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ABSTRACT 


The  purpose  of  this  work  is  to  formulate  and  solve  certain 
optimization  problems  arising  in  the  fields  of  engineering  economics, 
scarce  resource  allocation,  and  transportation  systems  planning. 

The  scope  and  structure  of  optimization  theory  is  presented  in  order 
to  place  subsequent  work  in  proper  perspective.  A  branch  and  bound  al¬ 
gorithm  is  rigorously  developed  which  can  be  applied  to  the  optimization 
problems  of  interest.  A  rounding  operation  is  defined,  which  provides  a 
powerful  rejection  rule  and  permits  the  calculation,  at  each  stage  of  the 
solution  process,  of  an  upper  bound  and  a  feasible  solution  in  addition  to 
the  usual  lower  bound.  ^This  double  bounding  technique  implies  little  or 
no  extra  computational  effort. 

Subsequent  chapters  are  devoted  to  the  study  of  various  cases  of 
capital  investment  problems.  Investment  in  sets  of  independent  project, 
is  considered  first.  For  the  (0-1)  multi-dimensional  knapsack  problem  a 
new  formulation,  interpreted  as  a  network  synthesis  problem  on  a  bipartite 
graph,  is  given.  This  formulation  permits  the  straightforward  application 
of  the  branch  and  bound  algorithm,  and  allows  the  solution  of  the  linear 
program  associated  with  each  node  of  the  solution  tree  to  be  obtained  by 
inspection. 

This  study  is  pursued  by  considering  capital  investment  in  a  single 
time  period  as  a  special  case  of  the  previous  problem.  Certain  economic 
interpretations  are  derived  by  investigating  the  dual  program  of  the  dis¬ 
crete  knapsack  problem.  A  parametric  branch  and  bound  method  is  developed 
which  permits  the  solution  of  the  knapsack  problem  for  a  range  of  values 
of  the  budget  ceiling. 

Two  formulations  are  proposed  for  a  special  case  of  deferred  capital 
investments,  referred  to  as  the  multi-knapsack  problem.  The  first  formu¬ 
lation,  after  a  transformation  by  means  of  a  model  equivalent,  leads  to  a 
branch  and  bound  algorithm  which  requires  the  solution  of  a  standard  trans¬ 
portation  problem  with  surplus  and  deficits  and  certain  routes  prohibited 
at  each  step  of  the  algorithm.  The  second  model,  although  it.  may  require 
a  larger  tree  before  optimality  is  reached,  permits  the  solution  by 
inspection  of  the  linear  program  associated  with  each  node  of  the  solution 
tree. 


The  final  part  of  this  thesis  studies  capital  investment  for 
dependent  proposals  in  the  context  of  urban  transportation  planning.  The 
branch  and  bound  algorithm  is  adapted  to  the  link  addition  network  design 
problem,  where  a  descriptive  traffic  assignment  model  is  employed. 

Finally,  fov  the  multistage  link-addition  network  synthesis  problem, 
a  normative  model  is  formulated  as  a  block-angular  mixed-integer  linear 
program.  A  partitioning  technique  is  employed  to  take  advantage  of  the 
highly-structured  form  of  the  model. 

We  conclude  with  a  detailed  presentation  of  the  partitioning 
technique  of  Benders,  as  applied  to  Doth  continuous  and  mixed-integer 
programming  problems  presenting  a  block-angular  structure. 
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BIOGRAPHICAL  NOTE 


CHAPTER  I 


FUNDAMENTAL  CONCEPTS  OF  OPTIMIZATION  THEORY 


1.1  INTRODUCTION 

The  goal  of  this  chapter  is  to  formalize  the  concepts  relevant  to 
describing  the  nature  and  scope  of  optimization  thzoKy.  We  begin  by  de¬ 
fining  the  optimization  problem  and  discussing  its  complex  nature.  We 
identify  the  fundamental  steps  in  the  solution  process  of  optimization 
problems  as:  i)  problem  definition,  ii)  formulation  of  an  optimization 
model,  iii)  selection  of  a  solution  method,  and  1v)  application  of  the 
solution  method.  Each  step  of  the  process  and  its  Implications  is  dis¬ 
cussed  in  detail  for  a  variety  of  applications. 

A  classification  of  optimization  models  and  of  solution  methods  Is 

presented.  The  material  covered  in  this  chapter  and  a  historical  survey 

* 

of  optimization  theory  (cf.  Appendix  B)  are  Intended  to  present  a  general 
framework  of  the  theory  which  will  be  applied  in  the  main  body  of  this  work 
to  specific  types  of  optimization  problems.  Finally,  we  shall  discuss 
some  important  aspects  of  optimization  In  the  context  of  analysis  and  de¬ 
sign  of  engineering  systems. 


1.2  THE  OPTIMIZATION  PROBLEM 

Whenever  an  engineer  or  decision  maker  Is  confronted  with  the  problem 
of  selecting  a  course  of  action  from  a  set  of  alternatives  he  will  be  com¬ 
pelled  to  choose,  from  the  available  alternatives,  the  best  in  terms  of  a 
certain  predetermined  goal  or  set  of  goals  relevant  to  the  nature  of  the 
prcalem. 

It  Is  assumed  that  the  degree  to  which  the  goal  or  objective  of  the 
problem  is  reached  for  each  alternative  course  of  action  can  be  evaluated 
by  a  quantitative  method.  In  other  words,  a  measure  of  the  utility  of  each 
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course  of  action  may  be  obtained,  allowing  the  decision  maker  to  select 
the  alternative  yielding  the  maximum  utility.  The  degree  to  which  the  goal 
is  obtained  is  the  {iguie  mtxcL  for  a  particular  solution. 

DEFINITION,  An  optimization  problem  is  defined  as  the  one  of  selecting 
among  a  set  of  various  alternatives  (possibly  infinite)  of  a  certain  prob¬ 
lem,  the  one  for  which  a  given  figure  of  merit  is  optimized  (i.e., 
maximized  or  minimized). 


1.3  OPTIMIZATION  THEORY.  THE  NATURE  OF  THE  OPTIMIZATION  PROBLEM 

The  nature  of  optimization  problems  Is  often  quite  complex,  and  a 
wide  variety  of  cases  presenting  different  characteristics  is  encountered 
in  practical  problems.  To  visualize  the  complexity  which  may  be  present 
in  the  nature  of  the  problem,  consider  the  following  examples:  1)  a  deci¬ 
sion  maker  may  be  confronted  with  a  problem  having  a  clearly-defined  ob¬ 
jective  to  optimize;  however,  the  problem  may  or  may  not  be  subject  to  a 
set  of  constraints.  He  may  also  have  to  consider  the  solution  to  the  prob¬ 
lem  on  the  assumption  of  either  deterministic  or  stochastic  behavior. 

11)  the  decision  maker  may  have  to  Interact  and  compete  with  other  parti¬ 
cipants,  each  of  whom  Is  attempting  to  make  decisions  which  optimize  his 
own  figure  of  merit,  ill)  several  decisions  may  have  to  be  made  on  a 
multistage  problem,  where  the  goal  sought  is  a  long-range  optimization  as 
opposed  to  suboptimization  of  a  particular  stage  of  the  problem. 

It  is  this  complex  nature  as  well  as  the  different  structural  char¬ 
acteristics  of  the  models  (cf.  Section  1.6)  that  clearly  Indicate  the  need 
for  a  variety  of  techniques  to  cope  with  the  solution  of  optimization 
problems.  The  set  of  all  these  techniques,  namely  those  included  under 
the  specific  names  of  mcLthmaticaZ  pA.ogfumyU.ng,  game.  thecAy,  itcutiiticat 
decJjtion  theoAy,  dtjnanUc  pAogAcontUng ,  contAoi  theoAij,  calctxiiu  o &  vasUa- 
tionA,  etc.,  constitute  with  their  theoretical  foundations  the  general 
theory  of  optimization, 

Optimization  theory  in  its  widest  sense  Is  the  unified  branch  of 
mathmaticat  anatijiis  that  provides  a  formal  approach  to  the  solution  of 
optimization  problems. 
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1.4  SOLUTION  PROCESS 

The  solution  process  for  optimization  problems  may  not  be  identical 
in  all  cases  and  may  differ  depending  on  the  special  nature  of  the  prob¬ 
lem;  nonetheless  it  will  always  be  possible  to  distinguish  In  the  process 
the  basic  steps  indicated  in  Fig.  1-1.  The  various  loops  indicate  possibl 
revision  of  the  previous  decision. 


Fig.  1-1.  Optimization  Problem  Solution  Process 


1.5  PROBLEM  DEFINITION 

At  the  problem  definition  stage  the  decision  or  control  variables 
governing  the  problem  are  identified,  and  the  form  of  interactions  among 
the  variables  is  specified.  A  figure  of  merit  must  be  defined  in  terms 
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of  the  relevant  control  variables  and  the  range  of  variation  of  the  controls 
must  be  explicitly  or  Implicitly  specified.  Finally,  the  constraints  to 
be  satisfied  by  the  variable  must  also  be  established. 


1.6  FORMULATION  OF  A  MATHEMATICAL  MODEL 

Once  the  problem  has  been  properly  defined,  the  subsequent  step  will 
be  to  formulate  an  abstract  model  (usually  a  mathematical  model),  that 
faithfully  represents  the  essential  structure  of  the  problem  and  that  may 
be  amenable  to  solution  through  application  of  a  well-known  procedure. 
Whenever  reference  Is  made  to  models  it  will  be  understood  in  the  sense  of 
Karlin*,  "a  model  is  a  suitable  abstraction  of  reality  preserving  the  es¬ 
sential  structure  of  the  problem  In  such  a  way  that  its  analysis  affords 
Insight  into  both  the  original  concrete  situation  and  other  situations 
which  have  the  same  formal  structure". 

It  Is  clear  that  solution  of  the  model  will  produce  accurate  results 
only  to  the  extent  that  the  model  is  representative  of  the  original  prob¬ 
lem.  If  the  problem  has  not  been  properly  modeled,  its  solution  may  lead 
to  dubious  results  or  completely  erroneous  ones;  for  instance,  consider 
the  case  of  a  linear  programming  model  giving  an  unbounded  solution  as  a 
result  of  a  constraint  of  the  problem  not  being  included  In  the  model. 

We  shall  now  analyze  some  distinctive  characteristics  of  optimization 
models  that  will  permit  their  convenient  classification.  This  will  be  use¬ 
ful  for  further  Identification  of  the  mooels  that  will  be  encountered  ir 
subsequent  chapters. 

We  shall  distinguish  three  main  components  of  an  optimization  model: 

1)  the  set  of  problem  variables,  ii)  the  figure  of  merit  to  be  optimized, 
Hi)  the  domain  of  definition  of  the  problem  variables  (determined  by  the 
constraints  of  the  problem).  The  optimal  solution  for  certain  classes  of 
optimization  problems  consists  of  numerical  values  taken  by  the  problen 
variables,  satisfying  the  constraints  and  simultaneously  nptimizlri  the 
figure  of  merit.  Other  classes  of  optimization  problems  seek  to  f\  d  a 


♦ 

Karlin,  S.,  Mcrfk<jnaXicat  MeXhodi  and  Tkiony  in  Gan&i,  PAognaming,  and 
Ec.ononu.0, ,  Vol.  J,  Addison-Wesley ,  1959,  p.  1 . 
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curve  or  function  (variational  problems),  that  satisfies  a  set  of  con¬ 
straints  and  renders  optimal  a  certain  functional  expression  of  the  set 
of  feasible  solut’on  curves. 

For  certain  problems  the  objective  will  be  amenable  to  a  closed  form 
mathematical  representation  as  a  function  of  the  control  variables.  For 
other  problems  this  closed  representation  might  not  be  obtainable,  and  the 
figure  of  merit  for  a  given  set  of  values  of  the  control  variables  may 
only  be  known  after  a  complex  process  has  been  completed  (such  as  a  simu¬ 
lation  process,  an  engineer'ng  analysis,  the  solution  of  an  elaborate 
computer  program,  or  a  table  look-up). 

Furthermore,  the  problem  may  be  const 'ained  or  unconstrained.  For 
constrained  problems  capable  of  formulation  in  a  closed  form  mathematical 
representation,  the  nature  of  the  constraint  expressions  may  be  quite  di¬ 
verse,  For  instance  they  may  be  algebraic  or  transcendent  expressions, 
equalities  or  inequalities,  linear  or  nonlinear  with  the  domain  of  the 
variables  being  a  discrete  set  or  the  continuum.  Also  some  of  the  con¬ 
straints  may  be  differential  equations  or  definite  integrals. 

In  the  light  of  the  above  discussion  we  have  developed  the  tree- 
structured  classification  of  optimization  models  illustrated  in  Fig.  1-2. 
The  tree  obviously  may  be  expanded  in  both  the  vertical  and  horizontal  di¬ 
rections  to  make  it  as  complete  as  is  needed  or  desired. 

We  sha’l  be  able  to  distinguish  certain  branches  of  the  tree,  repre¬ 
senting  specific  classes  of  problems,  for  which  the  solution  procedures 
form  a  well-established  mathematical  development.  Fcr  instance,  models  in 
the  constrained  optimization  branch  for  which  both  the  constraints  and 
objective  may  be  represented  in  closed  algebraic  form  constitute  that  part 
of  optimization  theory  generally  known  as  ma,thema.Uca,l  pAcgiantncng. 

As  a  second  example,  consider  the  class  of  problems  for  which  the 
explicit  objective  function  is  expressed  by  a  definite  integral  (functional 
objective)  with  or  without  subsidiary  conditions.  The  solution  of  such 
models  falls  within  the  scope  of  the  ci^itcut  ca.ic.uLit,  vtvuxtfuon* , 

Finally,  consider  those  models  with  constraints  and/or  objective 
lacking  a  closed  mathematical  representation  The  optimization  of  such 
models  must  be  attained  by  any  means  short  of  orute  force;  the  techniques 
usually  applied  fall  under  the  general  name  ot  wet  s catch  method*. 


CONSTRAINED 

MODELS 


8 


CONCEPTS  OF  OPTIMIZATION  THEORY 


An  example  of  this  class  would  be  a  certain  stochastic  process  (e.g.  a 
waiting  1 ; ne »  a  given  renewal  process)  being  analyzed  by  means  of  a  costly 
computer  simulation.  The  input  parameters  may  be  varied  and  the  simula¬ 
tion  executed  tor  each  set  of  values.  Associated  with  the  output  of  each 
run,  a  mzaauAfe  (MOE),  of  the  corresponding  input  para¬ 

meters  may  be  estimated.  If  the  problem  s  to  select  the  input  parameters 
that  optimize  the  MOE,  a  direct  search  techn  que  ij  required  in  this  case 
to  find  the  optimum  while  minimizing  the  number  of  simulated  trials. 


1.7  SOLUTION  TECHNIQUES 

Solution  techniques  are  the  procedures  and  algorithms  devised  for  the 
solution  of  optimization  problems.  The  actual  solution  usually  entails 
determination  of  numerical  values  of  the  control  variables  and  the  optimum 
value  of  the  figure  of  merit. 

Optimization  methods  are  usually  broken  down  ’nto  two  major  categor¬ 
ies:  <.ncLLwct  and  dLn.e.cA  mtxhcd*.  With  direct  methods,  the  optimum  solu¬ 
tion  is  sought  by  directly  calculating  values  of  the  objective  function  at 
different  points  of  the  feasible  domain.  The  values  thus  obtained  are 
compared  and,  by  means  of  an  a  xiliary  criterion,  a  new  point  is  next  anal¬ 
yzed  which  hopefully  will  improve  the  value  of  the  objective  function. 

Alternatively,  indirect  methods  look  for  a  set  of  values  of  the 
control  variables  that  satisfy  known  necessary  conditions  for  optimality. 
The  classical  method  of  the  differential  calculus  >?  an  example  of  the 
indirect  type.  In  effect,  values  of  the  ve"iables  are  sought  for  which 
the  first  derivatives  of  the  objective  function  vanish,  provided  that  con¬ 
tinuity  of  the  function  and  existence  of  derivatives  m  the  region  of 
interest  are  guaranteed  In  this  way,  the  optimization  problem  has  been 
transformed  into  a  root-finding  problem. 

The  Simplex  algorithm  of  linear  programming  exhibits  features  of  both 
the  direct  and  indirect  methods  It  performs  a  direct  search  over  extreme 
points  of  the  feasible  domain  only  (points  sat’sfying  the  necessary  con¬ 
dition  for  an  optimum)  in  such  a  way  that  the  objective  function  is  at 
least  as  good  as  m  tne  previous  step.  Finally,  the  optimum  among  the  set 
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of  extreme  points  Is  detected  when  the  indirect  criterion  of  feasibility 
of  the  complementary  solution  to  the  associated  dual  problem  Is  satisfied. 

For  certain  mathematical  models  of  optimization,  a  solution  method 
may  include  transform'ng  the  original  model  into  an  equivalent  one  that 
promises  to  be  more  tractate  than  the  former  (cf.  Chapters  III  and  IV). 
Consider  the  methodology  of  g±ynct>uc.  p>ug>um<.ng'H  ■.  in  this  case,  the 
polynomial  optimization  is  formulated  in  terms  of  its  dual  problem  and  this 
is  the  model  that  is  actually  solved  Another  example  is  the  transforma¬ 
tion  into  a  linear  programming  problem  of  a  nonlinear  program. 

Direct  techniques  may  be  subdivided  into  two  major  groups. 
imuUctAtOLu.  and  & .xl  methods.  Simultaneous  search  tecnniques  cal¬ 
culate  values  of  the  objective  function  or  response  surface  at  a  set  of 
points  determined  a  pm,oM  by  a  certain  search  strategy.  Sequential  search 
methods,  on  the  other  hand,  deal  with  sequential  examination  of  trial  solu¬ 
tions,  basing  the  location  of  subsequent  trials  on  the  results  of  earlier 
ones.  We  present  in  Fig.  1-3  a  subset  of  representative  solution  techniques 
for  each  one  of  the  classes  of  methods  discussed  in  this  section 


1.8  SELECTION  OF  A  METHOD 

The  select’on  of  a  convenient  solution  method  for  a  given  problem 
depends  on  the  type  of  model  employed,  the  existing  solution  techniques  for 
that  particular  model,  and  the  computation  facilities  available  to  the 
engineer-analyst 

In  the  selection  process  one  may  consider  such  factors  as  linearities 
of  the  model,  number  of  variables,  number  of  constraints.,  special  struc¬ 
tures,  separability  or  weak-couphng  of  variables  in  constraints  and/or 
objective,  objective  or  constraint  surfaces  of  readily  interpreted  geo¬ 
metric  character,  etc 

The  final  selection  of  a  well-suited  method  for  a  particular  problem 
depends  then  on  the  detailed  properties  of  the  model  as  well  as  the  solu¬ 
tion  techniques  that  form  part  of  a  software  package  of  an  available  computer 
instal lation. 


★ 

Duff  in,  R  J, ,  E.  L  Peterson,  and  C  M  Zener,  Geowfruc  P\ognamb\g, 
Joh”  Wiley,  1967 


DIRECT  METHODS 


FIG.  1-3  Classification  of  Solution  Methods 
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We  have  just  presented  a  brief  review  of  some  classes  of  optimization 
problems,  the  mathematical  models  applicable  to  these  problems,  and  the 
methods  available  for  their  solution,  To  place  the  developments  of  optlml* 
zatlon  theory  In  proper  perspective,  the  reader  is  referred  to  Appendix  B 
for  a  survey  of  the  most  significant  contributions  of  different  mathema¬ 
ticians  through  the  centuries. 

In  the  following  section  we  shall  discuss  concepts  relevant  to 
engineering  systems  optimization,  as  a  framework  for  the  class  of  problems 
undertaken  in  the  main  body  of  the  text. 

1.9  ENGINEERING  SYSTEMS  OPTIMIZATION 

The  engineer  uses  analytical  and  experimental  methods  to  analyze  and 
interpret  the  behavior  of  the  physical  world  in  such  a  way  that  appropriate 
decisions  can  be  made  regarding  Investment  of  scarce  resources  for  the 
development  of  facilities  of  economic  utility. 

In  general,  the  engineer  seeks  a  design  which  satisfies  a  certain 
specified  performance  of  the  facility  in  an  economical  manner.  The  mean¬ 
ing  of  econcnuaU  Is  subject  to  various  interpretations.  It  may  mean  a 
least  cost  design  Including  both  construction  and  operating  costs.  On  the 
other  hand,  one  may  seek  a  design  yielding  the  highest  level  of  performance 
consistent  with  the  given  construction  and  operating  budgets;  one  may  also 
mix  these  extreme  cases 

With  this  in  mind,  we  can  view  the  task  of  the  engineer  as  that  of 
providing  the  best  solution  to  the  problem  as  described;  therefore,  the 
engineer  confronts  an  optimization  problem  In  the  sense  discussed  In 
previous  sections. 

From  the  practical,  computational  point  of  view,  the  majority  of 
engineering  system  design  problems  are  sufficiently  complex  that  one  can¬ 
not  provide  a  mathematical  model  for  the  entire  problem  which  could  be 
solved  by  one  of  the  solution  techniques  indicated  previously. 

However,  any  engineering  system  design  problem  is  deflned  In  terms  of 
a  set  of  boundaries  which  delineate  the  range  of  the  systems  of  interest. 
These  boundaries  represent  an  arbitrary  but  presumably  reasonable  separa¬ 
tion  of  the  system  under  consideration  from  other  systems  In  which  It  is 
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Imbedded.  Hence,  the  design  problem  can  be  viewed  as  a  suboptimization  of 
a  set  of  subsystems,  the  union  of  which  compose  the  system  of  Interest. 

Therefore,  it  seems  perfectly  natural  to  fragment  an  engineering 
system  design  problem  into  components,  some  of  -.ihlch  may  be  sufficiently 
limited  as  to  permit  the  application  of  optimization  techniques.  It  Is 
evident  that  the  set  of  optimum  solutions  to  the  selected  components  will 
not  in  general  constitute  an  optimum  to  the  original  system  but  simply  a 
suboptimal  solution. 

The  traditional  process  for  solving  an  engineering  system  design  problem 
usually  takes  tne  form  of  a  trial  and  error  procedure.  However,  in  those 
cases  wner«.  s>si  i  />  pr^olems  are  su;:e>*?.il :/  subjected  to  mathe¬ 
matical  optimization  techniques,  the  engineer-analyst  draws  boundaries 
about  the  fragment  of  the  design  problem  so  that  a  closed  form  mathematical 
representation  of  the  system  is  obtained.  Known  optimization  techniques 
are  then  applied  to  this  representation  or  model,  and  an  optimal  solution 
to  the  design  problem  is  calculated. 

When  it  is  possible  to  isolate  a  system  fragment  of  significant  phy¬ 
sical  extensiveness  and  calculate  its  optimum  design  by  a  convergent  process, 
we  say  that  a  synthesis  algorithm  exists  for  the  design  of  the  system. 

While  it  may  not  be  possible  to  isolate  a  section  of  a  design  problem 
such  that  its  optimization  may  be  termed  a  synthesis  procedure,  one  ex¬ 
pects  to  find  parts  of  engineering  design  problems  whose  solutions  will  be 
small-scale  optimizations.  The  solution  of  these  small-scale  optimizations 
which  occur  as  parts  of  the  total  system  w:ll  be  of  special  interest  in  the 
incremental  process  of  developing  a  total  synthesis  algorithm 

Throughout  this  work  we  shall  be  concerned  with  exploring  parts  of 
engineering  design  problems,  the  solution  of  which  may  be  solved  by  known 
optimization  techniques.  The  first  part  of  the  material  covers  optimal 
allocation  of  capital  resources  to  a  finite  set  of  facilities.  Problems 
involving  synthesis  of  transportation  networks  will  be  developed  In  the 
remaining  parts  of  the  work. 
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CHAPTER  II 


A  BRANCH  AND  BOUND  ALGORITHM  FOR  A  CLASS  OF 
DISCRETE  OPTIMIZATION  PROBLEMS 

2.1  INTRODUCTORY  REMARKS 

Branch  and  bound  algorithms,  a  class  of  solution  methods  for  Integer 
programming  problems,  have  been  extensively  studied  since  the  first  pro¬ 
cedure  of  this  class,  offering  a  new  and  fresh  approach  to  the  solution  of 
combinatorial  problems,  was  published  by  Land  and  Dolg  [1]  In  1960. 

The  name  branch  and  bound  Is  due  to  Little  et  ol .  [2].  These  authors 
successfuly  employed  a  technique  In  this  class  to  obtain  a  solution  to  the 
traveling  saleman  problem  which  was  substantially  more  efficient  than 
solutions  previously  available.  This  result  encouraged  further  Investi¬ 
gation  Into  the  applicability  of  this  technique.  Improvements  of  existing 
methods  were  carried  out  by  Dakin  [3]  and  Driebeek  [4],  and  further  appli¬ 
cations  are  due  to  Ignall  and  Schrage  [5]  on  the  job  scheduling  problem, 
to  Efroymson  and  Ray  [6]  on  a  plant  location  problem,  Wershdorfer  et  al. 

[12]  on  the  assignment  of  numbers  to  nodes  of  a  tree-dimensional  grid  so 
that  the  bandwidth  of  the  associated  node-node  incidence  matrix  is  mini¬ 
mized,  and  to  Gavett  and  Plyter  [7]  on  the  optimal  assignment  of  facilities 
to  locations.  A  survey  on  the  state  of  the  art  up  to  1966  may  be  found  in 
the  work  of  Lawler  and  Wood  [9], 

Various  formalizations  of  the  general  class  of  branch  and  bound  methods 
have  been  undertaken  by  Agin  [8],  Lawler  and  Wood  [9],  Roy,  Nghlern,  and 
Bertler  [10]  and  others.  Most  recently  Ichbiah  [11]  generalized  the  work 
of  Roy,  et  al.  and  developed  a  parametric  branch  and  bound  technique. 

In  subsequent  chapters  we  snail  study  various  optimization  problems 
arising  in  the  fields  of  transportation  systems  analysis  and  design,  and 
capital  budgeting  for  independent  and  dependent  projects.  These  problems 
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will  be  mathematically  formulated  as  discrete-bivalent  programming  prob¬ 
lems  (l.e.,  one  In  which  a  pair  of  feasible  values  Is  specified  for  each 
member  of  a  subset  of  problem  variables).  These  problems  arise  In  appar¬ 
ently  Independent  areas  but  It  Is  possible  to  develop  mathematical  models 
for  these  problems,  which  In  fact  are  closely  related.  These  models  can 
all  be  solved  by  a  branch  and  bound  technique  of  the  Land  and  Dolg  type 
requiring  the  solution  of  network  flow  or  transportation  type  problems  at 
each  step  of  the  Iterative  procedure. 

Since  each  of  the  problems  we  shall  consider  Is  solved  by  a  variant 
of  our  branch  and  bound  technique,  this  chapter  presents  the  general  formu¬ 
lation  of  this  method  as  a  basis  for  the  particular  applications. 

The  problems  that  we  shall  study  share  the  characteristic  that  a 
feasible  solution  can  be  obtained  with  little  or  nc  computational  effort 
at  every  stage  of  the  algorithm.  Associated  with  this  property  Is  a  means 
for  developing  both  an  upper  and  a  lower  bound  to  the  objective  function 
at  each  stage  of  the  procedure.  This  double  bounding  technique  leads  to 
a  reduction  of  the  search  space  and  to  an  Increase  in  the  efficiency  of 
the  solution  technique. 

The  next  section  describes  the  mathematical  structure  associated  with 
our  class  of  problems,  and  subsequent  sections  describe  the  common  elements 
of  the  solution  technique  and  prove  Its  validity  and  finite  convergence. 


2.2  MATHEMATICAL  FORMULATION  FOR  THE  CLASS  OF  PROBLEMS 

Let  x  denote  a  vector  In  En  and  S^  a  closed  and  bounded  convex  set 
with  boundaries  defined  by  hyperplanes  In  En.  Let  T^  be  a  finite  non¬ 
empty  set  of  vectors  In  the  same  space,  and  denote  bv  0^  a  finite  subset 

of  Sj  obtained  by  the  intersection  of  S^  and  T^ ,  •-  Tj. 

Consider  the  following  discrete  optimization  problem  the  solution  of 
which  Is  to  be  obtained. 

P  :  Determine  x°  and  z°  so  a:  to 
Minimize  z  «  t'(x) 

Subject  to  x  e 

where  f  Is  a  single-valued  function  of  x. 
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DEFINITION.  Let  us  denote  by  Aj  the  j'th  auxiliary  continuous  problem, 
derived  from  P  as  follows: 

*  * 

Aj  :  Determine  £  (j)  and  z  (j)  so  as  to 
Minimize  z(j)  *  f(x) 

Subject  to  x  e  Sj  ,  j  -  1,2,.. 

For  j*1,  S1  Is  given  and  for  j>l,  Sj  is  a  subset  of  to  be  defined  In 
section  2.4.  We  shall  assume  that  a  finite  algorithm,  to  be  called  after 
Dakin  [3],  the  iub-algofuMm,  exists  for  solution  of  problem  A  j.  Further¬ 
more,  It  Is  assumed  that  a  feasible  solution  to  problem  P  may  be  determined, 
for  each  j,  by  "simple  Inspection"  cf  the  solution  to  problem  Aj.  This 
solution  will  be  denoted  by  £(j),  :.(j).  The  "inspection"  to  be  performed 
on  the  optimum  solution  to  A^  to  obtain  a  feasible  solution  to  P  will  be 
called  a  A.ounding  opesuUion. 


2.3  THE  DIRECTED  TREE 

The  branch  and  bound  algorithm  for  solution  of  problem  P,  to  be  set 
forth  In  section  2.5,  Is  an  Iterative  technique  that  may  be  Interpreted  as 
the  generation  of  a  directed  tree:  T(1)  *  [N( 1 ) ,  A(1)],  where  N(1)  and 
A(1)  are  respectively  the  set  of  nodes  and  the  set  of  directed  arcs  at  the 
end  of  iteration  1.  At  each  iteration,  except  for  the  first  one  during 
which  only  the  root  node  of  the  tree  is  created,  two  new  directed  arcs  and 
nodes -will  be  added  to  the  sets  N  and  A. 

Associated  with  each  node  jeN(1)  are  a  subset  fl.  of  G-j  and  a  subset 
Sj  of  S^,  and  associated  with  each  arc  (j,k)  e  A(1)  is  a  set  (cf. 
Section  2.4). 

At  the  end  of  the  1  * th  Iteration,  the  sub-ot  of  N(1)  corresponding 
to  the  terminal  nodes  of  the  tree  will  be  denoted  by  C(1).  The  set  C(1) 
will  be  partitioned  further  Into  three  subsets,  F(i),  E(1)  and  R(1)  such 
that  F(J  E  JJR  *  C.  Set  F  will  be  called  the  set  of  teatible  on  active 
noda,  E  the  set  of  in^eas-cbl c  on.  excluded  node i,  and  R  the  set  of 
rejected  ncde& , 

The  algorithm  starts  by  generating  the  root  of  the  tree,  node  1, 
associating  to  It  and  solving  A^ .  From  then  on,  and  in  an  iterative 
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fashion,  bifurcating  arcs  and  their  corresponding  nodes  are  added  to  the 
tree  according  to  a  bunching  operation.  These  directed  arcs  have  as 
origin  a  conveniently  selected  node  from  F(1).  For  each  node  j  thus 
created,  the  values  x  (j),  z*(j)  and  £{j),  z(j)  are  obtained  by  solving 
the  auxlliaty  problem  Aj  and  applying  a  funding  ope/iation  to  Its  optimal 
solution. 

This  Iterative  procedure  terml nates  when  the  solution  to  the  original 
problem  P,  or  sufficient  evidence  of  the  exlstencr  of  no  solution,  has 
been  obtained.  This  evidence  Is  given  by  the  operations  of  bounding,  ex¬ 
clusion  and  * ejection  (to  be  defined).  In  conjunction  with  the  branching 
and  rounding  operations  mentioned  above. 


2.4  BRANCH  AND  BOUND  OPERATIONS 


Let  flj,  a  non-empty  subset  of 


DEFINITION  1 .  Branching  Operation. 

(If  fi 

and  SjC  S1  be  the  sets  associated  with  node  j.  The  branching  operation 
Is  defined  by  a  partition  of  flj  Into  two  subsets  and  P.^  such  that: 


,j  vii  -  «,  no  branching  operation  will  take  place,  see  Definition  3), 


nrU  Vl  “ 
nrO  *V+i  *  * 


(2.1) 

(2.2) 


where  ♦  Is  the  empty  set.  This  partition  is  achieved  by  creating  two 
directed  arcs  (j,r)  end  (j,r+l)  emanating  from  node  j  with  associated  sets 
Vj  r  an<*  r+1  ant*  two  no<ks  r  ar,d  r+^  with  associated  sets  fir  and  0^ 
such  that: 


fljn  V4,r  *  “r 

n4^  VJ.r+l  ’  Vi 

He  observe  the  following  theorem  which  characterizes  V. 

J 


(2.3) 

"'d  Vj  ,r+1 : 
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THEOREM  2.1  Given  ftj,  Aa^ici&nt  e.ond±UoM  ^oft  Vj  f  and  Vj  ^  to  divine, 
a  paAtitioning  Aatii^ying  (2.1)  and  (2,2)  a/*;.: 

(2'4) 

fl3®»J>rUvJirtl)  U.5) 

Proof:  Assume  that  (2.4)  and  (2.5)  hold;  then  by  Intersecting  both  sides 
of  (2.4)  with  flj  we  obtain 

and  since  the  intersection  of  sets  is  distributive 

(sljnvj.r>n{njnvJ>rH.1)  -  * 

Hence  from  (2.3)  =  * 

Finally  (2.5)  is  equivalent  to 

ajn(Vj.rU  ’  flj 

or  «iJnvjiriu(ajnvjir+,)  =  !lj 

Hence,  from  (2.3)  firU  =  This  completes  the  proof. 

Next  we  associate  to  nodes  r  and  r+1  the  subsets  Sf  and  Sr+1 ,  defined 
as  follows: 


Vsj0  VJ.r 


Sr+1  =  Sj^Vj,r+l 


(2.6) 


From  the  results  of  lemma  2.1  below,  Oj  is  a  subset  of  Sj.  We  observe 
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that  only  the  following  condition  is  satisfied  for  Sf  and  S^: 

SrHSr+1  -  «  (2.7) 

That  Is,  the  sets  Sr  and  are  mutually  exclusive  although  they  may  not 
be  collectively  exhaustive  of  S^- 

After  a  finite  number  of  branching  operations  have  been  performed, 
we  mty  expect  to  have  generated  nodes  t  for  which  has  been  reduced  to 
a  single  element  of  the  original  domain  n^.  It  is  also  expected  that  the 
corresponding  St  Is  reduced  to  contain  exclusively  the  same  single  ele¬ 
ment,  so  that  fit  =  St<  We  observe  that  this  is  possible  for  since,  by 
hypothesis,  is  finite;  but  this  $  not  so  for  St,  since  S1  Is  infinite. 
Hence,  In  order  to  guarantee  that  eventually  =  $t  we  have  to  restrict 
further  the  sets  Vj  f  and  Vj.r+1  in  the  following  way:  it  is  assumed  that 
the  sets  Vj  f  and  Vj  r+1  are  such  that  in  a  finite  number  of  branching 
operations,  nodes  with  containing  one  single  element  have  an  associated 
St  containing  only  the  same  single  element.  In  the  case  of  the  particular 
applications  considered  in  the  present  work,  this  Is  a  relatively  simple 
condition  to  satisfy.  Finally,  to  initialize  and  make  possible  the  branch¬ 
ing  operation,  the  sets  and  are  assigned  to  the  root  node  of  the 
solution  tree. 

LEMMA  2.1  Let  tlr  and  Sr  be  the  aeta  auoci^d  with  node  r  o$  TU).  Then 
ie>  a  aubaet  o X  S  . 

Proof:  By  Induction.  In  effect,  for  r  =  1,  by  hypothesis.  Let 

us  assume  that  for  node  j,  is  satisfied.  Then  letting  r  be  the 

Immediate  successor  of  j  and  by  intersecting  each  side  of  (2.3)  and  (2.6) 
we  have 


arnsr  M«p  Sj)0  vj>r 

8ut  from  the  previous  assumption,  fijO  *  Oj  and  therefore 

arnsr.l)jn*jir 
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Or  using  (2,3),  fif  0  Sr  ^  flr,  or  equivalently  HrCsr,  which  completes  the 
proof. 

Note  that  for  a  terminal  node  j-t  with  St  containing  a  single  element 
x  such  that  x  e  »  then  £  S^. 

DEFINITION  2,  Bounding  operation.  Given  the  current  set  F(1)  of  active 
nodes,  the  bounding  operation  will  be  defined  by  means  of  the  following 
actions: 

a)  Lower  bounding  operation.  This  consists  of  selecting  the  node  ' 
k  e  F(1)  such  that 

z*(k)  =  min  {z\j)}  (2,8) 

jeF(i) 

and  of  setting  the  value  L^ ,  the  current  lower  bound  for  problem  P,  equal 
to  the  value  given  by  (2,&): 

Li  -  z*(k)  (2.9) 

Node  k  is  said  to  be  the  bounded  node  for  iteration  i,  defining  the  node 
from  which  branching  will  take  place  at  the  next  iteration. 

b)  Upper  bounding  operation,  This  consists  of  finding  the  value 

z(s)  -  min  {z(j)l  (2.10) 

jrF(i) 

which  constitutes  the  current  least  upper  bound  of  the  problem,  and  of 
setting  the  value  U  equal  to  the  value  given  by  (2.10): 

U1  --  z(s)  (2.11) 

Fxpressions  (2  9)  and  (2  n)  constitute,  respectively,  the  best  lowor  and 
upper  bounds  of  the  problem  at  the  end  of  the  i'th  iteration.  This 
statement  will  be  substantiated  by  means  of  the  following  lentnas. 
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LEMMA  2.2  1 5  node.  J  U  the  dmecUate  pA.edee.eM on.  r  then 

Z*(j)  <  z\r) 

it  *  *  dr 

Proof:  If  x  (j)  =  x  (r)  then  2  (r)  =  2  (j)  since  both  A,,  and  A,  share 
"  *  *  >  J 

the  same  objective  function.  Otherwise,  if  x  (j)  f  x  (r)  and  since 

S„CS,  due  to  the  way  the  branching  was  defined,  problem  A„  Is  more  re- 

strlcted,  and  consequently  z  (j)  <  z  (r), 

LEMMA  2  3  Let  k  be  the  bounded  node  o{  iteration  1  uiith  a64>oeiated  value 
L.|  given  by  [2  9)  *°,  z°  <$  the  optimal  iolutuon  to  P,  then  L -  <  z°. 

Proof:  Let  L(  and  L1+1  be  the  values  given  by  (2.9),  associated  with  any 
two  consecutive  iterations  From  lemma  2.2  and  since  branching  occurs 
from  the  last  bounded  node,  it  follows  that  L^  <  Li+1  and  thus  <  L^  , 
l  2  1.  Now  assume  that  the  process  of  branching  continues  until  the  entire 
tree  has  been  developed  at  iteration  t  -  t.  The  set  F(t)  will  contain  all 
nodes  associated  with  feasible  solutions  to  P  (guaranteed  by  branching 
operation).  Then  L+  *  min  [z  (j)]  *  z°.  Hence  L.  <  L«.  -  z°,  which 

1  4fC| fl  1  * 

completes  the  proof.  J  v  ' 

From  lemma  2,2  and  the  definition  of  bounding,  we  observe  that  at 
each  iteration  the  bounding  operation  indeed  gives  a  lower  bound  to  prob¬ 
lem  P  as  indicated  by  lenma  2,3;  and  also  a  better  lower  bound,  (closer  to 
the  optimum)  than  the  previous  iteration  as  asserted  by  lemma  2.2  and  the 
fact  that  branching  occurs  from  the  bounded  node  of  the  previous  iteration. 

LEMMA  2.4  i(j),  z(j)  o  the  feasible  solution  to  problem  P  obtained 

$ncm  a  Acundung  operation  at  node  j,  then  z°  <  z(j) 

Proof:  If  <jj)  £  *°  it  follows  that  z°  -  z(j).  Otherwise  x(j)  f  x°,  and 
since  x(j)  is  only  a  feasible  solution  to  P,  then  z°  <  z(j). 

The  rounding  operation  thus  provides  an  upperbound  on  z°  at  each  node 
It  is  performed  upon.  Now,  since  at  the  end  of  iteration  1  the  best 
lower  bound  corresponding  to  node  k  is  L1 ,  and  the  best  upper  bound 
corresponding  to  node  s  is  the  following  theorem  results: 
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THEOREM  2.2.  At  any  iteiation,  and  -  U^,  tkz  optimal  solution 

hai  been  obtained;  it  coiWApondi  to  the  bounded  solution  o£  node  s  o j  the 
cuAAent  it elation. 

Proof:  The  first  part  is  evident:  from  the  definition  of  bounding  and 
from  lemmas  2  3  and  2  4,  it  follows  that  L1  s  2°  i  U1  and  therefore 

Li  *  v 

It  remains  to  be  proved  that  if  L(  -  ,  node  s  is  an  optimum  solution. 

Additional  branching  would  make  the  lower  bound  greater  than  U^;  and  since 
a  feasible  solution  to  P  associated  with  node  s  has  already  been  found, 
all  other  feasiMe  soiuf’ons  to  P  rot  ye*:  discovered  would  yield  no  im¬ 
provement  in  z°  given  by  s.  Consequently,  the  feasible  solution  to  P 
associated  with  node  s  is  the  optimal. 

DEFINITION  3.  Exclusion.  The  exclusion  operation  is  defined  for  a  terminal 
node  r  of  C(i)  for  which  the  corresponding  set  £?r  is  empty.  Since  &r  is 
empty,  no  need  exists  to  consider  further  branching  from  node  r  and,  as 
part  of  the  exclusion  operation,  the  node  is  assigned  to  the  set  E(i)  of 
excluded  nodes. 

LEMMA  2.5  1$  the  solution  to  Ar  i&  in^eaiible,  then  =  4>. 

Proof:  If  A  is  infeasible,  its  domain  of  definition  is  empty:  =  4>. 

r  r 

From  lemma  2.1,  since  't  follows  that  s  4>. 

DEFINITION  4.  Rejection  The  rejection  operation  on  node  r  consists  of 
assigning  the  node  to  the  set  R(i)  of  rejected  nodes  if  the  following 
condition  is  satisfied: 


LEMMA  2.6  l ^  jot  node  r  at  iteration  l,  z  (r!  o  gieatei  than  the 
uppeAbound  at  the  pievioui  iteiaticn,  no  fiuxtke  t  blanching  £  row  t  o 
neeeAsaxy 
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Proof:  Consider  the  following  possible  cases: 

i)  x*( r)  t  and  z *{r)  >  U^:  Since  U^_,  Is  by  definition  an 
upperbound  of  the  problem,  it  follows  that  flr  does  not  contain  the  op¬ 
timum  solution  to  P,  and  no  further  branching  Is  required. 

11)  x*(r)  e  and  z*(r)  >  U^:  Although  node  r  Is  a  feasible 
solution  to  P,  the  sane  argument  as  for  case  1}  holds. 

DEFINITION  5.  Rounding  operation.  Let  x.  (j),  z  (j)  be  the  solution  to 
A.  associated  with  node  j.  The  rounding  operation  consists  of  obtaining 

J  ^  ^ 

from  £  (j),  z  (j)  a  feasible  solution  S(j),  z(j)  to  problem  P. 

For  the  classes  of  problems  considered  throughout  this  work,  unless 
otherwise  Indicated,  this  operation  is  possible  by  conveniently  rounding 
off  certain  components  of  x  (j).  When  this  operation  is  possible,  the 
double  bounding  feature  of  the  algorithm  may  be  employed,  thus  resulting 
in  an  improved  branch  and  bound  method. 

We  note  that  if  the  operation  is  possible  for  each  node  j,  then: 

a)  The  uppertound  may  be  updated  at  each  iteration,  thus  making 
possible  the  execution  of  the  rejection  operation.  Since  a  rejected  node 
Is  assigned  to  the  subset  R(i),  and  the  selection  for  branching  is  per¬ 
formed  among  the  nodes  in  subset  F(i),  no  further  information  associated 
with  the  rejected  node  is  required. 

b)  The  updating  of  the  upperbound  U.  at  each  iteration  reduces  the 
interval  of  uncertainty  of  tho  optimal  solution  z°  at  each  iteration, 
since  L^  <  z°  s  U..  Furthermore,  if  the  branch  and  bound  method  is  used 
for  suboptimization,  and  the  process  is  terminated  before  an  optimal 
solution  has  been  obtained,  the  algorithm  nonetheless  provides  valuable 
Information  at  that  step  In  effect,  the  available  information  is 
represented  by  a  feasible  solution  to  the  original  problem  P,  p » us  a  lower 
bound  on  the  problem  that  permits  us  to  estimate  how  far  the  available 
feasible  solution  is  from  optimality. 

c)  A  measure  of  effectiveness  for  the  rounding  operation  is  provided  by 
the  algorithm.  Note  that  the  set  C(i)  of  terminal  nodes  of  iteration  i 
contains  exactly  i  nodes  This  is  true,  since  t\.  each  iteration  two  new 
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nodes  are  created,  r  and  rtl  and  the  node  j  from  which  branching  occurred 
is  no  longer  terminal,  hence  the  net  Increase  is  one  terminal  node. 
Furthermore,  according  to  the  partition  of  C(i)  defined  earlier,  each 
terminal  node  is  assigned  to  one  of  the  sets  F(i),  E(i)  or  R(i). 

Thus,  if  we  let  a  be  number  of  elements  in  F(1)  and  3  the  number  of 
elements  in  R(i),  a  measure  of  effectiveness  (MOE)  of  the  rounding 
operation  may  be  defined  as 

MOE  ■>  B 


Witn  tne  possible  operations  and  associated  lemmas  established,  we 
now  proceed  to  describe  the  algorithm 

2.5  SPECIFICATION  OF  THE  ALGORITHM 

Thebranch  and  bound  algorithm  consists  of  an  initial  step  that 
generates  the  root  of  the  greeted  tree  (iteration  1),  plus  subsequent 
analogous  iterations,  continued  until  either  the  optimal  solution  or  suf¬ 
ficient  evidence  of  the  existence  of  no  solution  is  obtained.  Note  that 
under  the  assumption  that  the  rounding  operation  is  possible,  P  will  al¬ 
ways  have  a  feasible  solution. 

STEP  1.  Set  i - 1  an'i  create  node  j*l.  Set  F(l)-E(l )-R{l )-4>  Solve  A^. 

If  the  solution  is  infeasible,  stop;  problem  P  has  no  solution. 

Otherwise,  if  x*{l)  e  ,  stop;  the  solution  is  optimal.  If 

*  * 

x  (1)  t  n,,  bound  node  1  with  L^  ^  z  (1).  Round  node  1  to  ob¬ 
tain  x(l),  z(i)  Set  U,  -  z(l).  If  L1  -  Uj,  stop;  the  rounded 
solution  is  optima'  Otherwise,  l1  <  Assign  node  1  to  F(l), 
set  i  -  i  tl  and  go  to  step  i 

STEP  l  a)  BRANCH  Branch  from  bounded  node  j  t  F(i).  Delete  node  J 
from  F(i)  Create  nodes  r  and  rtl  and  directed  arcs  (j,r)  and 
(j,  r*l)  Solve  problems  Ar  and  Ar+^,  ar.d  in  both  cases  do  the 
following;  if  Af  (Art^)  is  infeasible,  exclude  node  r(r+l)  by 
assigning  it  to  EiO  Otherwise  Ar.  (A  ?5  has  an  optimum  solution. 
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*  * 

If  2  (*•)  (z  (r+l))  >  U,  , ,  izjict  node  ru*1)  oy  assigning  It  to 

«  A 

set  R(i)  Otherwise,  z  (r)  (z  (rtl))  <  U^,  so  assign  node 
r ( r+ 1 )  to  set  F(i). 

b)  ROUND  Round  node  r(rfl)  if  it  was  assigned  to  either  F(1) 
or  R(  1 ) . 

c,I)  BOUND  FROM  ABOVE,  Set  U1  =  ml r.  [Ui  l ,  z(r),  z( r+l )3  for 
nod-  r(rvl)e  F(i)  or  R(i)<  Reject  nodes  of  F(1)  having 
z  (J)  >  Ut  by  assigning  them  to  R(i). 

c  2)  BOUND  FROM  BELOW.  Select  node  k  e  F(l)  by  using  lower 
bound  operation.  Lower  bound  node  k  with  L^  =  z  (k).  If  L^  *  U^, 
stop;  the  feasible  solution  that  provides  the  upperbound  is 
optimal.  Otherwise,  Lj  <  U^.  Set  1  =  i  +  1  and  go  to  step  1. 

It  remains  to  be  s^own  that  the  algorithm  indeed  finds  the  optimal 
solution  in  a  finite  number  of  steps.  Since  It  Is  assumed  that  the  round¬ 
ing  operation  is  possible,  a  feasible  solution  to  P  exists,  and  therefore 
an  optimal  solution  exists.  Moreover,  from  the  way  the  branching  operation 
has  been  defined  and  the  hypothesis  that  is  finite,  the  algorithm  would, 
in  a  finite  number  of  steps,  generate  all  feasible  solutions  to  P.  (i.e., 
solutions  corresponding  to  terminal  nodes).  And  finally,  from  theorem 
2-2,  the  optimal  solution  may  be  identified. 
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CAPITAL  INVESTMENT  ON  INDEPENDENT  PROJECTS 

3.1  THE  CAPITAL  ALLOCATION  PROBLEM 

We  shall  refer  to  the  problem  of  optimally  allocating  a  fixed  capital 
budget  imon^  a  nmte  set.  of  competing  proposals  as  the  capital  aZlocatton 
pzeblejv* .  We  can  make  a  basic  distinction  between  two  classes  of  alloca¬ 
tion  problems  which  will  result  in  substantially  different  analytical 
formulations  and  hence  different  techniques  to  be  used  in  their  solution 
process.  These  correspond  to  the  cases  of  independent  and  of  dependent 
investment  proposals.  We  shall  consider  as  independent  projects,  after 
Lorie  and  Savage  ([1],  p.  229),  those  for  which  "the  worth  of  individual 
investment  proposals  is  not  profoundly  affected  by  the  acceptance  of 
others". 

In  this  and  in  the  following  chapter  we  shall  be  concerned  with 
optimal  allocation  of  resources  among  independent  proposals,  while  in 
subsequent  chapters,  optimal  capital  allocation  for  dependent  projects 
will  be  studied  for  various  problems  in  the  context  of  transportation 
network  synthesis. 

Special  cases  of  the  capital  allocation  problem  have  been  studied  by 
Lorie  and  Savage  [1]  for  the  case  of  independent  projects.  They  first 
consider  the  problem  of  allocating  a  fixed  amount  of  money  among  competing 
alternatives,  each  requiring  a  given  capital  outlay  in  a  single  time 
period.  The  objective  to  be  optimized  is  the  sum  of  the  net  present  values 
of  the  investments  (i.e.,  the  algebraic  sum  of  positive  and  negative  costs 
flows  discounted  to  the  present,  using  the  firms  "cost  of  capital"  as-  the 
discount  rate).  Their  proposed  solution  method  is  based  on  ranking  the 

j~ - 

Although  the  discussion  in  this  chapter  is  in  terms  of  money  allocation, 
it  is  in  fact  applicable  to  .*1  location  of  a  variety  of  other  scarce  resources. 
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investment  proposals  In  decreasing  order  of  present  value  per  dollar  of 
outlay  required,  and  accepting  them  in  that  order  until  the  fixed  budget 
is  exhausted.  They  do  not,  however,  deal  with  the  ranking  of  various 
combinations  of  projects  and  therefore  their  method  does  not  guarantee  an 
optimal  solution. 

Lone  and  Savage  also  consider  the  case  where  projects  require  capital 
outlays  in  several  time  periods,  and  they  propose  a  method  later  shown  by 
Weingartner  [2]  to  suffer  from  several  serious  defects.  Weingartner  in 
[2]  identifies  capital  rationing  as  an  optimization  problem  and  develops 
an  Integer  programming  model.  This  model,  for  the  single  period  case, 
corresponds  to  the  Dantzig  formulation  of  the  (0-1)  kna.p6a.ck  problem,  [3]. 
The  model  employed  by  Weingartner  in  the  multiple  outlay  case  corresponds 
to  the  (0-1)  multi- cUmau,  tonal  knapiack  problem  (i.e.,  the  knapsack  prob¬ 
lem  with  restrictions  on  weight,  volume,  height,  etc.).  Traditionally, 
the  knapsack  problem  has  been  solved  by  dynamic  programming  and  most 
recently  by  an  enumerative  technique  developed  by  Gilmore  and  Gomory  [4], 
For  the  multidimensional  knapsack  problem,  Weingartner  and  Ness  [5]  use 
a  recursive  relation  to  solve  the  complement  problem  (where  projects  are 
successively  eliminated  Instead  of  accepted)  and  have  reported  interest¬ 
ing  computational  results.  Shapiro  and  Wagner  [6]  have  also  studied 
these  problems,  demonstrating  their  connection  with  renewal  problems  formu¬ 
lated  by  means  of  recursive  expressions. 

Cord,  [7]  formulates  the  single  period  problem  for  the  case  of 
uncertain  returns,  and  seeks  to  maximize  the  total  return  on  investment 
while  maintaining  the  average  variance  for  the  total  investment  within  a 
certain  predetermined  value.  Cord  uses  the  method  suggested  by  Bellman 
[8]  of  incorporating  one  constraint  Into  the  objective  function  by  means 
of  a  Lagrange  multiplier  and  then,  with  a  single  constraint  left,  applying 
the  dynamic  programming  solution  of  the  knapsack  problem.  A  discussion 
of  the  drawbacks  of  the  method,  and  the  example  problem  of  Cord,  may  oe 
found  In  [9]. 

Finally,  we  point  out  that  the  present  discounted  value  used  by 
Lorie  and  Savage  and  by  Weingartner  has  been  a  controversial  issue  due 
to  the  interest  rate  or  “cost  of  capital"  employed  to  obtain  such  dis¬ 
counted  values.  Baunol  and  Quandt  [10]  have  indicated  the  serious 
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difficulties  that  this  approach  entails,  and  have  suggested  an  alternative 
objective  function  based  on  explicit  discount  rates  and  subjective  utili¬ 
ties.  Throughout  this  work,  we  shall  assume  subjective  utility  functions 
to  express  the  corresponding  figure  of  merit  of  the  models  to  be  derived. 


C.2  THE  VARIOUS  CASES  OF  INVESTMENT  DECISIONS 

We  shall  consider  various  cases  of  investment  decisions  on 
independent  projects  confronting  a  fi  rm  or  a  government  agency.  We  shall 
derive  programming  models  in  each  case  which  may  be  interpreted  as  network 
flow  problems  on  capacitated  networks,  with  the  additional  constraint  that 
flow  on  a  subset  of  the  arcs  must  be  either  zero  or  the  upperbound  on  the 
arc.  The  solution  techniques  provided  are  special  cases  of  the  branch  and 
bound  algorithm  presented  in  Chapter  II.  The  problems  to  be  analyzed  and 
their  characteristics  are  the  following: 

i)  The  capital  investment  problem  requiring  cash  outlays  in 
various  time  periods  for  each  project  is  formulated  as  a 
maximum  flow  problem  on  a  single-source  single-sink  capacitated 
network,  where  flow  on  the  arcs  represents  cash  flow  and  the 
flow  on  the  arcs  emanating  from  the  source  is  restricted  to 
be  either  zero  or  at  upper  bound.  Its  analogy  to  a  special 
class  of  plant  location  problems  is  indicated.  The  branch  and 
bound  algorithm,  as  adapted  to  the  problem,  permits  the  use  of 
the  rounding  operation;  furthermore,  the  solution  of  the  linear 
programming  problem  associated  with  each  node  of  the  solution 
tree  may  be  obtained  by  simple  inspection. 

1i)  The  (0-1)  knapsack  problem  is  then  considered  as  a  special 
case  of  the  previous  problem.  The  solution  proposed  by  Lorie 
and  Savage  (that  of  maximizing  net  present  discounted  value) 
is  shown  to  represent  the  root  node  of  the  branch  and  bound 
tree. 

Next  part  will  be  devoted  to  analysis  of  multistaged 
resource  allocation  problems  where  the  horizon  and  staging 
are  assumed  to  be  given. 
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111)  The  ft of  these  problems  to  be  considered  is  a  capital 

budgeting  problem  requiring  a  single  costs  outlay  per  project 
and  subject  to  capital  rationing  at  each  period;  but  the  cash 
outlays,  and  thus  the  investment  decision,  may  be  deferred  to 
a  late'  period 

Tne  resulting  model  which  we  shall  call  the  nutti- 
fenapaacfe  problem  is  studied,  ce'tam  of  its  properties  deter¬ 
mined,  and  fina(>y  an  equivalent  network  flow  model  on  a 
bipa'tite  graph  is  derived  which  resembles  the  fixed-charge 
transportation  problem  considered  in  Chapter  VI,  The  branch 
and  bound  technique  as  applied  to  the  problem,  permits  the  use 
of  the  rounding  operation;  the  linear  program  to  be  solved  at 
each  node  of  the  solution  tree  is  a  capacitated  transportation 
problem  with  surpluses  and  deficits  and  with  certain  routes 
prohibi ted, 

iv)  Finally,  a  special  type  of  multi-knapsack  problem  is  considered 
in  which  all  items  (projects)  must  be  assigned  to  knapsacks  of 
given  capacity  so  as  to  minimize  the  number  of  knapsacks  re¬ 
quired  to  adequately  allocate  the  items  of  the  problem. 

Problems  in)  and  iv),  although  presented  within  the  framework  of 
capital  budgeting,  arise  in  a  variety  of  fields  and  in  particular  two  such 
applications  to  optimal  allocation  of  compute'  system  facilities  are 
discussed  in  detail 

3.3  THE  MULTIPERIOD  CAPITAL  INVESTMENT  PROBLEM 

Consider  a  government  agency  or  a  corporate  division  confronted  with 
the  problem  of  allocating  a  multi-staged  budget  with  ceilngs  on  each  stage, 
among  a  set  of  independent  projects  requiring  capital  outlays  in  various 
time  periods  (Problem  i)  Government  agencies  typically  face  this  prob¬ 
lem  when  the  available  amount  of  capital  is  determined  exogenously  by 
legislature  appropriation  or  by  government  budget  planners.  In  the  case 
of  a  corporate  division,  top  management  may  determine  the  budgets  and 
simultaneously  cut  ore  the  division  from  acquiring  additional  funds  from 
the  capital  market 
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Let  B.,  j=l,.  ,n  be  the  budget  ceilings  at  each  stage  or  time  period 

J 

and  let  a  >  0  be  the  capital  outlay  required  by  project  i,  (i=l,...,m) 

'  J 

at  time  period  j.  Assume  a  cortain  utility  f  associated  with  the 

*  ' 

acceptance  of  project  i  ,  the  t .  might  for  example  be  subjectively  deter¬ 
mined  by  the  decision  maker.  A  set  of  projects  must  be  selected  for 
investment  so  that  the  total  utility  is  maximized  while  maintaining  the 
capital  outlay  at  each  stage  within  the  corresponding  budgetary  ceiling. 

We  derive  an  analytical  model  by  considering  a  bipartite  network 
G  =  [N-j,  Np,  A],  where  N1  is  a  set  of  m  nodes  each  representing  a  project 

proposal,  anci  N«  a  set  of  n  nodes  each  associated  with  one  of  the  stages 
c  n 

considered  Let  t  a  be  the  "demand"  or  input  associated  with  node 
j-1  J 

i  e  N1  and  the  "demand"  or  output  associated  with  node  j  e  N2.  Let 

x...  the  flow  on  the  arc  (i,  j)  e  A,  represent  a  capital  outlay,  and 

capacitate  these  arcs  with  the  upper  bounds  a  .  Then  the  capital  allo- 

^  J 

cation  problem  defined  above  may  be  expressed  as  follows:  find  a  flow 
pattern  on  the  network  so  as  to 


n 


P'  :  Maximize 

2  - 

E  fi  yi 
i  =  l  1  1 

(3.1) 

Subject  to 

m 

I 

i=l 

x.  <  B 

U  J 

,  J-1.  • 

,n  (3.2) 

n 

l 

J-1 

n 

’J  J-1 

V  - 

,m  (3.3) 

0 1 

x  <  a 
ij  ’  iJ 

,  *  (i  j) 

c  A  (3.4) 

>1 

integer 

(3.5) 

where  the  yf  a*e  decision 

variables  associated  with 

each  node  i  e  N, 

which  may  take  on  th-.  values  0  or  1  according  to  whether  project  i  is 


f.  would  represent  the  net  present  value  of  investing  in  project  i, 
discounted  by  the  appropriate  rate  of  interest,  if  the  total  present 
value  approach  of  Lorie  and  Savage  is  :J.pted. 
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rejected  or  accepted  for  investment-  (By  summing  (3.4)  over  j  and 

comparing  the  result  with  (33),  we  obtain  y.  2  a^  *  2  a44;  and  since 

i  j U  js1  U 

by  (3.5)  y1  is  restricted  to  be  integer,  it  follows  that  the  only  possible 
values  for  y<  are  0  or  \  ) 

Constraints  (32)  restrict  the  capital  investments  incident  on  node 
j  to  be  within  the  available  budget  at  period  j.  Constraints  (3.3)  indi¬ 
cate  that  if  project  i  is  accepted  (yi  *•  1)  the  sum  of  the  flows  leaving 
node  1  must  be  equal  to  the  total  investment  required  for  that  project 
over  the  entire  horizon  Coupled  with  the  upperbounding  constraints  (3.4), 
this  condition  forces  the  flow  on  arcs  emanating  from  i  to  be  at  upper 

bound  as  expected,  By  the  same  reasoning,  if  y,  =  0,  then  E  x..  =  0  and 

1  i=i  'J 

the  flows  on  the  arcs  x^  are  at  zero  level.  J 

Observe  that  the  capital  budgeting  problem  as  interpreted  in  this 
network  flow  context  corresponds  to  a  special  class  of  plant  location 
probilems  [11],  [12];  however,  in  our  problem  we  are  maximizing,  the  flows 
from  plants  (projects)  to  destinations  (time  periods)  are  capacitated, 
and  there  is  no  explicit  participation  of  the  x^  in  the  objective 
function. 

Note  that  relations  (33)  permit  P'  to  be  exclusively  expressed  in 
terms  of  the  set  of  variables  x  . ,  as  follows: 

'  J 


P  :  Maximize 


m  n 

z  =  E  l  c,  x.. 
i*l  j--i  1 


(3.6) 


Subject  to 


x,j  s  Bj  .  3-1.. 


(3.7) 


0  5  X1J  "  *lj  *  ¥  ^  E  A  (3-8) 

y.  *  2  x  /  E  a  integer  (3.9) 

1  J«1  ,J  j*l  ,J 

n 

where  c.  *  f  /  E  aSl  represents  the  total  utility  of  project  i  per 
1  1  j*l 

unit  of  investment,  and  thus  all  arcs  emanating  from  the  same  node  i 
incur  the  same  cost  c1 
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We  observe  in  passing  that  if  constraint  (3.9)  is  deleted,  the 

resulting  problem  may  be  decomposed  into  n  mutuary  independent  programs 

of  the  form  Max  z-  ~  2  c,  x  ,,  2  *iiiBil0ii,,<ai  ,  f,  each 
J  1  1  J  ym  \  '  J  J  1  J  '  J  * 

one  associated  with  time  period  j;  the  solution  of  which  may  be  obtained 

by  simple  inspection  as  will  be  shown  later. 

Before  proceeding  to  develop  a  solution  technique  for  problem  P,  we 

shall  show  how  the  problem  may  be  formulated  as  that  of  obtaining  the 

maximum  flow  that  maximizes  total  utility. 

The  bipartite  network  G  with  multiple  sources  and  sinks  may  be 

transformed  into  an  equivalent  network  with  a  single  source  and  a  single 

sink.  This  may  be  done  by  adding  artificial  nodes  s  and  t,  and  artificial 

arcs  (s,i),  Y  i  e  N,  and  (j,t)  ¥  j  e  N0,  with  the  following  associated 

1  n  *■ 

values:  cs)  -  0,  u$]  -  £  and  c^t  *  0,  Ujt  *  ;  where  u$1  and  u^t 

j  "  1 

denote  the  upperbounds  on  the  respective  arcs.  Furthermore,  we  shall 
require  that  flow  on  arcs  (s,i),  t  e  N-j  be  either  zero  or  otherwise  that 
it  saturates  the  arc.  The  associated  network  is  shown  in  Fig.  3-1.  The 
first  number  on  each  a^c  represents  cost  and  the  second  represents  arc 
capacity, 


FIG  3-1 
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The  problem  therefore  may  be  expressed  in  terms  of  network  flow 
theory  as  an  anaZyi-U  problem:  find  the  maximum  flow  from  s  to  t  that 
maximizes  cost  on  the  network  of  Fig.  3-1,  as  well  as  Its  distribution 
pattern,  such  that  arcs  (s,i)  are  either  not  used  or  saturated. 

The  highly  combinatorial  nature  of  the  problem  does  not  permit 
network  flow  theory,  In  Its  current  state  of  development,  to  provide  a 
labeling  technique  (primal-dual  method)  to  cope  with  such  a  problem. 
However,  since  a  duality  theory  for  discrete  programming  has  recently 
been  developed  by  Balas  [13],  a  generalized  concept  of  complementary 
slackness  may  be  derived  for  this  class  of  problems  and  thus  a  generali¬ 
zation  of  the  out-of-kilter  method  [14]  for  networks  with  bivalent  arcs 
may  be  developed.  The  author  has  been  working  on  such  an  approach,  but 
Is  unable  at  this  point  to  present  final  successful  results. 


3.4  DEVELOPMENT  OF  A  SOLUTION  METHOD 

We  shall  adapt  In  this  section  the  branch  and  bound  algorithm 
presented  In  Chapter  II  as  applied  to  the  solution  of  problem-P.  The 
notation  to  be  employed  complies  with  that  used  In  Chapter  II.  We  shall 
first  define  the  sets  S-,,  T-j  and  as  follows: 

VCxu/J1xusV0jVs*ij:i  (3'10) 


[X 


ij 


/  2  x*4  *  2  a. r  *.  Integer,  x,  .*0  or  a...  ¥{1,j)]  (3.11) 
j-1  ’J 


«  S, n  Tj  (3.12) 

We  observe  that  the  sets  thus  defined  satisfy  the  assumptions  made 
In  the  original  development.  The  set  Sj  Is  a  closed  convex’  set  In  E0*11 
obtained  as  the  Intersection  of  the  hyperplanes  (3.7)  and  (3.8);  It  Is 
also  bounded  since  each  variable  x^,  from  (3, 8),. is  bounded  above  and 
below.  T.]  Is  a  non-empty  set  In  the  same  space  as  (e.g. 
x1j  *  0,  ¥  (1 ,  j)  e  T.,) ,  and  Is  also  finite  since  x^  ■  0  or  a^j. 
Finally,  from  (3,12)  Is  finite,  since  T1  Is  finite. 
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Note  also  that  since  a^  z  0  and  Bj  z  0,  at  least  a  feasible  solution 
exists,  namely,  the  a-totua  quot  l.e.  the  policy  of  aero  Investment;  and 
thus  an  optimal  solution  always  exists. 

E 'lancJUng  Operation.  Given  a  certain  node .1  of  the  solution  tree 
with  associated  sets  and  S^,  the  branching  Is  defined  by  their  Inter¬ 
section  with  the  sets 

V  T  [xu  ''  xkj  •  °-  V*  ^  • 0 

vt,r+i  ’  txU  /xkj  •  v  V  ■=•  *k  ■  1 

for  a  given  i  =  k.  The  sets  thus  defined  satisfy  the  sufficient  conditions 
to  form  a  partition  of  (cf.  theorem  2.1): 

't.rO  1  *  CX1 J  '  xkj  *  °*  xkj  ’  V  V  ■  *  <3',5> 

V.rU't.rt  ‘  tx13  '  xkj  '  0  or  akj’  V  <3-,6> 

and  since  is  a  subset  of  (by  branching  operation)  and  from 
(3.10)  to  (3.12),  the  variables  xkj  in  may  take  on  the  values  0  or  akj. 
Thus  the  intersection  of  with  (3.16)  Is  and  the  second  condition  for 

sufficiency  is  also  satisfied. 

Finally,  since  at  each  branching  operation  n  variables  are  set 
either  to  zero  or  at  upper  bound,  and  since  the  number  of  variables  Is 
finite,  eventually  we  will  obtain  a  terminal  node  t  with  St  =  con¬ 
taining  a  single  element  of  the  domain  and  hence  all  feasible  solutions 
to  P  may  be  enumerated  in  similar  fashion  by  developing  the  entire 
solution  tree. 


(3.13) 

(3.14) 


3.5  THE  AUXILIARY  PROBLEM  AND  ITS  SUBALGORITHM 

At  each  Iteration  of  the  branch  and  bound  algorithm,  associated  with 
each  newly-generated  node  Z  of  the  solution  tree,  a  continuous  auxZZZtAy 
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pnobtm  A^  derived  from  P  must  be  solved.  Denote  by  IQ  CN^  the  subset 
cf  nodes  of  the  network  G  for  which  y1  •»  0,  (le,(  nodes  representing 
rejected  projects);  by  ^  CN,  the  subset  of  N(  associated  with  y^  -  1, 
(accepted  projects);  and  by  R|  ;  N|  -  i0  •  1^,  the  subset  of  projects 
that  remain  "free"  to  be  accepted  or  '■ejected  at  this  step  of  the  solution 


process. 

Then  The  euxi iia-y  problem  A^ 

tat  is  the  form 

,  m  n 

•  Maximize  z  (£)  -  Z  Z  c. 

i*1  j'-l  1 

13.17) 

m 

Subject  to  L  x.  .  <  B 
i  =  i  ’>1  J 

r  J"  1  » • ^  * 

(3.18) 

o  i  x,j  <  a,j 

,  v-  (i  »j) 

(3.19) 

J=1  j=l 

a!j  "  0  ’  1  e  lQ 

(3.20) 

n  ,  n 

y,  -  r  x  .  /  E 

•  J-l  1J  j=l 

a1j  *  1  •  1  e  h 

(3.21) 

This  problem  is  a  linear  program  whose  solution  may  be  obtained  by 


* 

inspection.  Indeed,  from  (3.20),  x  - 
*  1 J 

xij  “  aij*  1  e  1  rne  Prob1e,n  (3.17), 

free  variables  ma/  do  decomposed  into  n 
the  form; 

Maximize  z,  -  Z  c .  x, , 

J  leR.  1  ,J 

i 

Subject  to  z  x  .  v  0 
ieR1  ‘  J 


0,  i  e  10^  From  (3.21)  and  (3,19), 
(3  18),  (3.19)  with  the  remaining 
muXiwliy  ■wde.pcr.de.ni  programs  of 

(3.22) 

(3.23) 


(3.24) 
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where  B-  -  B. 

J  J 


E  a.,.  We  may  assume  without  loss  of  generality  that 
i  el  j  1J 


the  indexing  of  projects  i  e  N]  is  done  in  decreasing  order  of  their 

utility  per  unit  of  investment  ,  so  that  >  c2  >  .. .  >  cm.  Under  this 

assumption  it  is  obvious  that  the  optimal  solution  to  (3.22)  -  (3.20)  may 

be  obtained  by  sinfply  setting  the  variables  x.^  equal  to  their  upper  bound 

in  the  order  of  the  index  i  until  the  budget  B^  is  exhausted.  If 

l  a,,  >  B.,  then  one  single  variable  will  take  on  a  value  less  than 
ieN-j  1J  ] 

its  upperbound,  The  problem  (3,22)  -  (3.24)  may  be  solved  for  all  j  in 
this  fashion.  Thus  the  optimal  solution  co  the  auxiliary  problem  A»  will 

'it 

be  x,  =  a,,  if  a,  .  <  6.,  zero  otherwise  and: 

Ij  1J  U  -  J 


.  if  r  e  I 

0 

,  if  r  £  ^ 


■ if  j,  xij '  Bo and  xij 5  V  r  c  Ri  • r  > ' 

1-1  1-1  (3.25) 

r-1  *  r  * 

,  if  l  x.  .  <  B.  and  Y.  x.  .  >  B.,  r  e  N,  ,  r  >  1 

i-,1  'J  J  ,.llj  J  I 


r-l  * 

,  if  Y  x  >  B..  re 
i=I  1J  J 


,  r  >  1 


The  objective  function  will  have  the  following  value: 


f  +  l  Ec  x  . 

’  led,  J  ’  'J 


(3.26) 


Note  that  if  for  an>  problem  B.  <  O.the  corresponding  A.  is 

J  c 

infeasible  and  the  node  of  the  tree  may  be  excluded  without  furthe- 
computation. 
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Rounding  opiAat^n.  Observe  that  from  the  optimal  solution  (3.25), 
(3.26)  to  A^,  and  from  (39),  the  set  of  y^  may  be  determined.  If  all 
of  them  are  integer,  then  ^3  25),  (3.26)  constitute  a  feasible  solution 
to  problem  P.  if  this  is  not  the  case,  then  a  feasible  solution  to  P  may 
be  obtained  without  any  additional  computational  effort  by  simply  setting 


*  * 

y.  ,  if  y.  *  Oor  1 
0  ,  if  0  <y*  <  1 


(3.27) 


H  ^ 

or  equivalently,  if  0  <  x.  <  a.  ,  then  set  x. .  *  0  for  all  j  e  N?, 

otherwise  set  x.  =  x.  .  The  value  of  tne  objective  function  is  given  by: 
■  J  *  J 


z  U) 


V 


i 

ieF^ 


(3.28) 


The  solution  thus  obtained  is  feasible  for  P;  note  that  in  obtaining 

/V  * 

x^j,  the  values  x^  have  been  reduced  if  changed  at  all,  hence  constraints 
(3.7)  and  (3.8)  are  still  satisfied.  Also  from  (3.27),  constraint  (3.9) 
is  satisfied  and  the  solution  is  feasible  fo»*  P. 

The  rounding  operation  defined  by  (3.27)  and  (3,28)  will  .efore 
permit  us  to  perform  rejection  of  certain  branches  of  the  branch  and 
bound  tree,  since  (3.28)  constitutes  a  lower  bound  on  the  optimal 
solution  to  P. 


3.6  THE  BRANCH  AND  BOUND  ALGORITHM 

Having  shown  that  the  assumptions  of  the  branch  and  bound  algorithm 
are  satisfied  and  having  developed  a  subalgorithm  for  solution  of  the 
auxiliary  problem,  we  may  now  proceed  to  establish  the  solution  method  as 
applied  to  our  capital  budgeting  Problem.  Note  that  we  have  simplified 
the  statement  of  the  algorithm  (cf.  Chapter  II)  and  also  have  expressed 
it  in  tenr-s  of  a  maximization  problem. 
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STEP  1.  Set  i  «  * ,  gene^te  node  1  bj  solving  Aj  (i.e. r  P  without 

constraints  (3.9/),  and  ‘ec  i  ,  be  the  optimal  solution. 

From  ;3  9)  obtain  *  .  If  ail  y.  are  0  or  i ,  stop;  the  solution 

*  * 

is  optimal  Otherwise  oi+'ii  node  1  with  Uj  *  z  .  Round  node  1 

to  obtain  (?,  5  ),  Set  L-j  *  z.  If  -_5  =  UJf  stop;  the  rounded 

solution  is  optima!.  Otherwise  Lj  <  !J,  Set  i  -  i  +  1  and  go 
to  step  i. 

STEP  1  a)  BRANCH.  3'ancn  from  bounded  node  l.  Select  one  y*  having 
a  fractional  value.  Create  nodes  r  and  r  *  1  and  directed  arcs 

(l,f)  and  H,r*  i).  Solve  problem  Ar  with  yk  -  0,  adding  k  to 

set  I  ,  and  solve  problem  A^.  with  y^  *  1,  adding  k  to  set  I-j. 

If  z  (rj  or  z  (r»i)  <  L:  te/ect  the  corresponding  node.  If 

one  is  infeasible,  zxtlxdi  the  corresponding  node. 

b)  ROUND  Round  nodes  r  ana  r  v  l. 

c.l)  BOUND  FROM  BELOW.  Set  L.  *  max  z(r),  2(r+l)]. 

Reject  all  nodes  w<th  z  <  L{. 

c . 2)  BOUNO  FROM  ABOVE.  Select  node  l  such  that  z *{l)  = 
max  {z  (k)},  fo<"  current  terminal  nodes.  Upperbound  node  l  with 

U*  *  z*(£).  if  L1  *  U-,  stop;  the  feasible  solution  that  provides 

the  lower  bound  is  optimal.  Otherwise,  <  U^-  Set  i  =  i  +  1 
and  go  to  step  i . 

*r 

At  each  branching  operat’on,  one  of  the  current  functional  y1  must 
be  selected  to  take  on  the  »a‘-ues  0  and  1.  Tnere  may  be  several  such 
variables  and  in  general  there  is  no  clear  cut  selection  rule  that  would 

guarantee  the  fastest  convergence  to  the  optimum.  Usually  certain  heuris¬ 

tic  rules  are  discovered  when  sufficient  computational  experience  with  the 

algorithm  is  available,  Our  experience  with  problems  solved  by  hand  has 

« 

indicated  that  selection  of  the  fractional  yf  having  the  largest  total 

investment  I  a  _  tends  to  result  in  infeasible  nodes  for  the  branch  y  =  1, 
j  rj  r 

thus  reducing  the  number  of  terminal  nodes  for  which  data  must  be  preserved 

and  resulting  m  a  reduced  time  of  computation. 
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3.7  SOLUTION  OF  AN  EXAMPLE  PROBLEM 

Consider  as  an  example  the  problem  given  In  Table  3-1,  taken  from 
[2],  Involving  10  projects  and  2  stages.  The  budgetary  ceilings  are 
B1  *  50,  R2  -  20 


Project 

No 

d,l 

di2 

f 

1 

£a  . 
j  ,J 

Project 

No 

fl 

ail 

di2 

fi 

?«1J 

J  J 

1 

15 

6 

2 

1,875 

6 

40 

30 

35 

0.615 

2 

17 

6 

6 

1.417 

7 

12 

18 

3 

0.571 

3 

15 

6 

7 

1,154 

8 

17 

54 

7 

0.279 

4 

12 

6 

6 

1.000 

9 

14 

48 

4 

0.269 

5 

14 

12 

3 

0  933 

10 

10 

36 

3 

0.256 

OLE  3-1 

To  generate  the  root,  (node  1),  the  auxiliary  problem  A^  must  be 
solve'4.  The  solution  is  obtained  for  each  time  period  by  means  of 
expressions  (3.25).  Here  I  *  I,  •  ♦  and  8.  *  B  .  The  solutions  are: 

xi  i  *  [6,  6,  6,  6,  12,  14,  C,  0,  0,  0] 

*.(2  =  £2,  6,  7,  5,  0,  0,  0,  0,  0,  0],  thus 

y,  »  [1,  I,  1.  M,  12,  14,  0,  0,  0,  0],  z*(l)  «  47  ♦  30.82  «  77,82 
T?  B  ST 

* 

and  by  rounding  y  *e  obtain 

^“[1.1.1.  0,  0,  0,  0,  0,  0,  0],  2(1 )  <47 

Therefore,  Lj  -  47,  Uj  -  77.82  and  node  1  is  bounded. 
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The  subsequent  iterations  and  their  pertinent  data  are  shown 


Step 

i 

feasible  nodes 
F{») 

Excluded 

nodes 

Rejected 

nodes 

Lower 

bound 

Li 

Upper 

bound 

ui 

1 

* 

1 

47 

m 

2 

4 

3 

2 

47 

EH 

3 

4*  ,  5 

59 

74.46 

n 

5  .  6  ,  7 

*  n 

:  v 

73.35 

mm 

5  ,  8*  ,  9 

7 

70 

73.59 

RB 

5  ,9  ,10* 

11 

70 

73.13 

7 

5*  ,  9  ,13 

12 

70 

73.1 0+ 

8 

9  ,  13  ,  15* 

14 

70 

72.96 

9 

9  ,  13  ,  17* 

16 

70 

72.78 

10 

9  ,  13* 

18,  19 

70 

71.68* 

11 

9 

20 

21 

70 

70.56* 

12 

4 

23 

22 

70 

70.54 

13 

25* 

24 

70 

70.51 

14 

♦ 

27 

26 

70 

70 

*  :  bounded  node 

t  :  search  on  a  new  branch  of  the  solution  tree 


TAciLE  3-2 


in  Table  3-2  and  the  atual  solution  tree  in  Figure  3-3  A  total  of  27 

nodes  are  generated  although  only  20  need  be  evaluated  by  means  of 

expressions  (3  27),  (The  infeas ib' 1 i ty  of  excluded  nodes  is  detected  when 

S.  <  0  for  any  j)  The  second  column  of  Table  3-2  indicates  the  number 
3 

of  current  terminal  nodes  for  which  information  must  be  stored  for  later 


use.  Note  that  at  any  one  time  no  more  than  three  such  feasible  terminal 
nodes  exist 
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Fig.  3-2  graphically  shows  the  effectiveness  of  the  rejection  rule. 

The  difference  between  the  lines  a  and  b  may  be  attributed  to  the  selec¬ 
tion  rule  employed  in  choosing  the  fractional  variable  y^  to  be  fixed  at 
each  iteration  The  difference  between  lines  b  and  c  represents  the  effect 
of  the  rejection  rule,  which  In  this  case  not  only  dampens  the  growth 
rate  of  the  set  or  feasible  nodes,  but  In  a  certain  interval  makes  the 
size  of  the  set  decrease  with  the  number  of  iterations.  Finally,  note 
that  since  the  number  of  feasible  terminal  nodes  at  the  end  of  Iteration 
14  is  only  one;  this  indicates  that  the  optimal  solution,  accept  projects 
1 ,  2,  4,  5  and  7,  is  unique. 


Step  1 


a  :  terminal  nodes  of  the  solution  tree 

b  :  terminal  nodes  excluding  infeasible  nodes 

c  .  terminal  nodes  excluding  infeasible  and  rejected  nodes 


FIG  3-2 


In  table  3-2  we  have  indicated  the  steps  at  which  a  search  along  a  new 
branch  of  the  tree  is  started  This  type  of  information  is  valuable  in 
the  context  of  computer  implementation  of  the  algorithm.  In  fact,  at 
each  iteration,  a  search  over  all  currently  feasible  nodes  must  be 
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performed  to  determine  the  node  from  which  to  branch  next:  max  (z*(k)}. 

keF(i ) 

This  Is  essentially  an  optimization  problem,  solved  by  a  table  look-up 
equivalent  to  an  exhaustive  search.  If  at  each  Iteration  the  node  numbers 
of  the  best  and  second  best  (or  as  many  as  desired)  i  values  are  stored 
separately,  and  updated  at  each  iteration,  the  number  of  table  look-ups 
is  reduced.  It  is  easy  to  verify  that  a  table  look-up  Is  not  required 
until  after  the  number  or  tree  branches  so  far  developed  at  least  equals 
the  number  of  decreasmgly  best  nodes  stored  separately  At  this  point, 
the  table  look-up  would  produce  the  next  set  of  decreasingly  best  nodes 
needed  to  begin  the  ne»t  cycle  of  the  operation 

1c 

In  the  example  considered,  if  the  three  values  with  the  best  z 
values  are  available,  the  first  table  look-up  would  be  required  at  itera¬ 
tion  12  to  determine  that  node  23  should  be  the  one  from  which  to  branch 
next.  At  that  point,  of  course,  the  table  consists  of  one  single  element. 

It  may  be  worthwhile  to  point  out  that  the  dimensionality  of  the  tree 
depends  largely  on  the  numbe.-  of  projects  considered  rather  than  on  the 
number  of  time  periods;  the  latter  only  implies  extra  computation  of  ex¬ 
pressions  (3.25)  for  all  time  periods  at  each  node  o*  the  tree.  That  is, 
the  total  number  of  nodes  of  the  final  tree  is  expected  to  vary  slightly 
as  a  function  of  the  number  of  time  periods  considered. 

Excellent  results  have  been  obtained  with  this  branch  and  bound 
technique.  For  a  report  on  this  computational  experience,  the  reader  is 
referred  to  [15]. 


CAPITAL  INVESTMENT  ON  INDEPENDENT  PROJECTS 


47 


3.8  NOTES  TO  CHAPTER  Ill 

[1]  Lorie,  J.  H  ,  and  L  J.  Savage,  Three  Problems  in  Rationing 
Capital",  rhe  Jratnat  Be^ineba.  Vol.  XXVIII,  October  1955, 
pp.  229-239. 

[2]  Weinga^tne*- ,  H  M  ,  Math  anxt-.c.a(  P-ugiami-utg  and  the  Analgbib  of 
Capita?  Budging  Pxjbieau  P'enti ce-Ha 1 1 .  Inc.,  Englewood  Cliffs, 

1963. 

[3]  Dantzig,  G  8  "Discrete-  Variable  Extremum  Problems" .  Opeoavtionb 
Resect". 'A,  April  1957,  pp,  766-277. 

[4]  Gilmore,  P  0  and  R.  E.  Gomory,  "A  L’near  Programming  Approach  to 
the  Cutting  Stock  Problem  -  Part  II",  Opeiatccnb  RebeaAc.k,  Sept.  - 
Occ,  ,  1963  ,  pp  863-888 

[5]  Weingartner,  H.  M.  and  D.  N.  Ness,  'Methods  for  the  Solution  of  the 
Multi-dimensional  0/1  Knapsack  Problem",  Ope.ruiti.enb  Re^eaAch,  Jan.  - 
Feb.,  1967 s  pp.  83-103. 

[6]  Shapiro,  J  F  and  H.  M.  Wagner,  "A  Finite  Renewal  Algorithm  for  the 
KriaDsack  and  Turnpike  Models",  Ope-xaticm  Research,  March-April, 

1967,  pp  319-341 

[7]  Cord,  J  ,  A  Method  for  Allocating  Funds  to  Investment  Projects  when 
Returns  are  Subject  to  Uncertainty',  Ma»taga>:c«t  Scioice,  January, 

1964,  pp  335-341, 

[8]  Bellman  R. ,  Comment  on  Dantzig's  Paper  on  Discrete-Variable 
Extremum  Problem-:",  Cpaatienb  Rebcaxcli.  Oct  1557  ,  pp  723-724. 

[9]  Weingartner,  H.  M. ,  Capital  Budgeting  of  Interrelated  Projects: 
Survey  and  Synthesis",  ‘lacagcmcnf  Science,  March  1966,  pp.  485-516 

[10]  Baumol  ,  W.  J.  and  R.  t  Quandt  "Mathematical  Programing  and  the 

Discount  Rate  under  C vital  Rationing  ,  tco*\n-«c  Joutnac,  June,  1965, 
pp.  317  329. 

[H]  Balinski,  M  L  ,  "Integer  Programing  Methods  Uses,  Computation”, 
Ma-uxuLmccf  S;,c::c  November  1965,  pp.  286-290. 


48  CAPITAL  INVESTMENT  ON  INDEPENDENT  PROJECTS 

[12]  Efroymson,  M.  A.  and  T.  L.  Ray,  "A  Branch  and  Bound  Algorithm  for 
Plant  Location",  OpcAot com  Reteatch,  May-June,  1966,  pp.  361-368. 

[13]  Balas,  E.,  "Duality  in  Discrete  Programming",  Technical  Report  67-5, 
Opemtcom  Rvseatch  House,  Stanford  University,  August  1967. 

[14]  Fulkerson,  D.  R.,  :;An  Out-of-Kilter  Method  for  Minimal  Cost  Flow 
Problems",  Journal  o ^  the  Society  Industrial  and  Applied 
Mathematic* ,  V.  9,  1961,  pp.  18-27. 

[15]  Ochoa-Rosso,  F.  and  K.  G.  Follansbee,  A  Stanch  and  Bound  Algorithm 
fat  the  Multiperiod.  Capital  Investment  Problem,  Research  Report 
R68-40,  Massachusetts  Institute  of  Technology,  Department  of  Civil 
Engineering,  July  1958. 


CHAPTER  IV 


SINGLE  STAGE  INVESTMENT:  THE  KNAPSACK  PROBLEM 

4-1  THE  (0  1)  KNAPSACK  PROELEM 

The  mo  1 11 -dimensional  knapsack  problem  studied  in  Chapter  III,  for 
the  special  cases  n  =  i  (one  single  period)  corresponds  to  the  (0-1) 
knapsack  problem  formulated  by  Oantzig.  As  we  mentioned  before,  this 
problem  has  tradi tional ly  been  solved  by  dynamic  programming.  Gilmore 
and  Gomory  fl]  have  recently  developed  a  theory  for  kna^ack  fiunctioM 
(i.e.,  the  opinion,  solution  z°(B)  is  a  knapsack  function  of  the  budget, 
which  is  considered  as  a  parameter).  Although  they  nave  presented  al¬ 
gorithms  for  the  knapsack  problems  without  upperbound  on  the  variables, 
for  the  (0-1)  case  they  have  only  indicated  how  an  algorithm  based  on 
dynamic  programming  should  be  derived  to  soive  the  problem  for  various 
values  of  the  budget. 

In  this  section  and  forthcoming  sections  we  shall  study  some 
important  economic  interpretations  related  to  the  dual  of  the  knapsack 
problem.  In  this  context,  we  shall  first  relax  the  discrete  restriction 
on  the  variables,  and  later  we  shall  also  formulate  the  dual  of  the  ori¬ 
ginal  knapsack  problem,  making  use  of  Balas'  discrete  programming  duality 
theory.  Two  brancn  and  bound  solution  methods  will  be  proposed  based  on 
the  genera1  algorithm  of  Chapter  ft.  The  f  it  method  constitutes  a 
special  case  of  trie  a'gonthm  developed  for  the  multi-dimens’ona'  knapsack 
problem  m  Section  3  6.  We  shall  discuss  the  simplifications  resulting 
r com  assuming  an  investment  horizon  of  one  time  period.  The  second  al¬ 
gorithm,  a1 though  a  branch  and  bound  solution  method  complying  with  the 
general  theory  of  Chapter  II.  is  closely  related  to  the  .uiftfv.  c  ug.-i ,  fkm 
of  F. alas  [2]  In  addition,  this  algorithm  is  oa -.,!»>«•  fs<,- ,  in  the  seise 
indicated  by  Ichbiah  [3],  (i.e.,  the  tree  s  >mui  taueour-ly  solves  the 
knapsack  problem  tor  various  values  of  the  nudee*. 1 
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4.2  ECONOMIC  INTERPRETATION  OF  THE  DUAL  LINEAR  PROGRAM 

By  collapsing  the  index  j  in  the  formulation  P  of  Section  3.3,  the 
knapsack  problem  of  Dantzig  may  bt.  modeled  by  the  following  discrete 
(bivalent)  programming  problem: 


Q  :  Maximize 

m 

Z  *  t  Cj  X. 
1*1  1  1 

(4.1) 

Subject  to 

m 

I  x,  <B 

1*1  1 

(4.2) 

0  s  x.  <  a1 

•*1 

(4.3) 

x.j  =  0  or  a.j 

•*1 

(4.4) 

where  (4.3)  is  obviously  redundant,  but  has  not  been  removed  here  for 
reasons  that  will  become  evident.  Let  us  assume  that  constraint  (4.4)  Is 
relaxed;  that  Is,  projects  may  be  accepted  for  which  the  capital  outlay 
Is  crly  a  fraction  of  the  total  capital  required,  a^.  Under  this  assump¬ 
tion  ^he  resulting  problem,  which  we  shall  denote  Q',  is  a  linear  program 
whose  associated  dial  program  is  the  following  problem  O': 


Minimize  z1  *  B  v  +  ?  a.  u. 

1*1  1  1 

(4.5) 

Subject  to  v  +  U|  > 

(4.6) 

v,  u1  >  0  ,  ¥1 

(4.7) 

where  v  Is  the  dual  variable  or  shadow  pvu.ce  associated  with  constraint 
(4.2)  and  the  uH  the  shadow  prices  associated  with  constraints  (4.4). 

Let  us  assume  that  =  f^  /  a^  >  0,  and  that  as  before,  the  c^  are 
ordered  in  decreasing  sequence,  (l.e.,  c1  >  >  ...  >  c^).  Under  these 

assumptions  It  Is  obvious  that  the  optimal  solution  of  (J  may  be  determined 
recursively  by 
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t  if  a-j  <  B  , 

,  if  a1  >  B  , 

,  if  x*  <  B  and  E  <  B,  r  >  1  (4-8) 

i-1  1  i-1  1 

r-1  *  r  * 

,  if  E  x.  <  B  and  E  x?  >  B,  r  >  1 

i=l  1  1*1  1 

r-1  *  , 

,  if  E  x  >  B,  r  :*  1 

i-1  1 

Denoting  by  r  *  s  the  index  of  the  last  accepted  project,  we  see 

from  (4.8)  that  only  x,  may  be  less  than  a,,  and  thus  project  s  is  the 

s  s 

only  one  partially  accepted.  The  rest  are  totally  accepted  if  r  <  s  or 
totally  rejected  if  r  >  s.  From  (4.8),  the  optimal  solution  will  satisfy 
(4.2)  as  a  strict  equality  (the  constraint  is  active). 

if  it  'k 

If  x  and  v  ,  u^  art  optimal  solutions  in  their  respective  programs, 
from  duality  theory  of  linear  programing  the  following  c.mpt'imzntany 
tlackneAA  conditions  must  be  satisfied: 

v*  [B-  2  x*]  =  0  (4.9) 

i-1  ? 

u*  [arx*]  -  0  (4.10) 

I  1  1 

X*  [u*  ♦  v*  -  Cl]  -  0  (4.11) 

*  m  * 

From  (4.9),  if  v  >0  then  E  x.  3  B,  which  is  the  case  under 

i*l  1 

consideration.  Ther  we  conclude  that  for  an  optimal  solution,  the  budget 

♦ 

ceiling  is  a  scarce  resource  and  v  may  be  interpreted  as  the  value  or 
imputed  rate  of  an  additional  dollar  added  to  the  budget.  Note  that  this 
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* 

rate  is  always  positive  In  the  linear  programming  case,  and  Its  value  will 
be  determined  below, 

A  * 

from  (4.10),  u.  *  0  for  rejected  projects.,  and  for  positive  uH  the 
project  is  accepted.  The  values  represent  the  Internal  rate  of  return 
of  one  dollar  invested  In  project  i.  From  (4.11),  for  accepted  projects 
the  following  holds; 

u|  !  ■  /  >  0  ,  U  s  (4.12) 

it  it 

We  shall  now  proceed  to  determine  the  optimal  solut.on  v  ,  £  to  the 

dual  problem.  For  an  optimal  solution,  both  objective  functions  are 

*  * 

equal,  and  taking  into  account  the  fact  that  =  0  and  u^  *  0  for  1  >  s, 
(l.e.,  for  rejected  projects)  we  obtain: 


s-1  ,  „  -  s?l 

l  i.  +  c  (B-  I  a.)  *  B  V  +  £  a, 

1*1  1  1  s  1*1  1  1*1  1 


* 


(4.13) 


By  substituting  (4.12)  In  (4.13)  and  solving  ftr  v*  the  following  condition 
results: 


v*  *  c$  (4.14) 

and  thus  the  optimal  value  of  an  additional  dollar  added  to  the  budget  Is 
equal  to  the  net  present  value  per  dollar  invested  of  the  last  project 
accepted  for  investment  in  the  optimal  solution  to  Q-  Finally,  substi¬ 
tuting  (4  14)  oack  In  (4.12), 

U1  *  ci  ~  S  ,  iss  (4.15) 

which  will  be  non  negative  since  we  have  assumed  c^  *  c$  for  i  <  s.  The 
rate  of  return  of  one  dollar  invested  In  project,  1,  1  <  s,  Is  the 


ft  *  ffl 

Obviously  v  *  0  for  the  trivial  case  E  a,  <  B. 

1*1  1 
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difference,  if  any,  between  the  net  present  values  per  dollar  invested  of 

project  1  and  project  s  (the  marginally  accepted  project). 

★ 

The  values  ul  suggest  a  natural  way  to  define  Investment  priorities. 
Lorit  and  Savage  [4]  propose  a  ranking  strategy  based  on  decreasing  net 
present  values  per  dollar  of  outlay,  that  is  in  decreasing  order  of  their 
c^.  Thus,  the  ranking  suggested  by  (4,15)  and  the  one  of  Lone  and 
Savage  would  result  in  the  same  projects  selected  for  investment.  We 
emphasize  again  that  this  holds  only  for  situations  tn  which  the  marginal 
project  accepted  may  be  accepted  as  a  fraction.  We  shall  consider  now 
the  dual  of  problem  Q,  for  which  attempting  project  divisibility  is  an 
absolute  pa 


4.3  THE  OUmL  OF  THE  DISCRETE  PROGRAM 

In  this  section  we  shall  study  the  dual  of  the  all -bivalent  program 
Q  based  on  the  duality  theory  of  discrete  programming,  [5],  We  shall  re¬ 
concile  this  theory  with  the  dual  program  suggested  by  Weingartner  [6] 
for  the  single  period  investment  problem. 

Let  us  consider  Q  subject  to  constraints  (4.2)  and  (4.4)  only,  and 
drop  the  redundant  constraints  (4.3)  from  further  consideration.  The 
dual  of  such  program  is  the  following  max-min  problem: 

D  :  Max  Min  z'  -  B  v  -  ?  $ .  x 
xv  i-l  1  1 

Subject  to  v  -  sl  -  cf  (4.16) 

v  >  0  ,  x,  -  0  cr  a}  ,  ^ 

Sj  unrestricted 


Since  the  slack  variables  si  are  unrestricted,  they  effectively 
nullify  constraints  (4  16)  and  problem  D  may  be  rewritten  as 


D  : 


m 

Mu,  j  8  v  -  Mav  (1  (v 

v  l  -s0 


A 

1 


(4  17) 
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where  s  *  {*  /  x,  >  o  or  a(),  As  suggested  by  Balas,  the  solution  of 

problem  Q  may  be  obtained  by  solving  the  equivalent  problem  (4.17)  using 

the  partitioning  technique  of  Benders  (cf.  Appendix  A),  thus  obtaining 
*  * 

the  optimum  x  and  v  However,  we  shall  assume  here  that  an  optimal 

* 

solution  k  to  problem  Q  is  already  available  (obtained  for  example  by 

the  branch  and  bound  technique  cf  Section  3.6). 

The  saddle-point  theorem  of  Balas  guarantees  that  if  an  optimal 
*  *  *  *  * 
solution  x  ,  2  ,  to  Q  exists,  then  an  optimal  solution  v  ,  s ,  z'  to  D 

★  if 

exists,  with  z  »  z'  . 


»  »  «  * 

THEOREM  4.1  (CorAplementaiy  Stactenciil,  1$  x  ,  z  and  v  ,  z’  axie  optimal 
bolutLon*  to  Q  and  V  nupectcuety,  then 


9 

V 


[8  - 


m 

E 


i* 1 


*  0 


(4.18) 


Proof:  Since  by  the  saddle  point  theorem  z  »  z‘  ,  we  have 


S  c.  x,  ■  B  v 
1-1  1  1 


1-S 


or  0 


*  _ 
v  [B 


m 

E 

1«1 


* 

So  for  v  >  0,  the  budget  is  a  scarce  resource.  Alternatively,  if 

the  budget  Is  a  free  good,  ?  x,  <  B  and  therefore  v  *  0.  The  dual 

1«1  1 

* 

variable  v  may  be  interpreted  as  In  the  continuous  case:  It  Is  the  value 

or  imputed  rate  of  an  additional  dollar  added  to  the  budget. 

*  * 

Once  an  optimal  solution  x  ,  z  to  Q  is  available,  then  the  optimal 

dual  variables  may  be  determined  as  follows:  If  ?  x  <  B,  then  from 

1-1  1 

*  * 

theorem  41  it  follows  that  v  *  0  and  thus  s^  *  -  c^. 
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m  *  Hr 

If  on  the  other  hand,  Z  a,  i  B,  then  by  substituting  x,  for  x.  In 

Ml  11 

D  we  obtain  the  following  square  system  of  equations  of  order  m  +  1: 

(4.19) 


*  m  *  *  » 

B  v  -  l  s.  x  =  z 


10 


i  1 


★  * 

*  '  Si  ■'  Ci 


, 


v  >0 


(4.20) 

(4.21) 


*  m  * 

LEMMA  4,1  Foa  cut  o pUmal  aofatUM  x.  to  Q,  iatii^xAg  E  x  ■  8,  the 

■C  JL 

iquoXion  (4.19)  it>  wdundant . 


Proof*  Multiplying  each  member  of  (4,20)  by  xi  and  summing  over  1  we  have 

★  m  *  m  *  *  m  -  x* 

v  E  Xj  -  Z  Sj  x  *  E  "1  1 

1*1  1  1*1  1  1-1 


m  * 

but  this  expression,  under  the  assumption  that  Z  x^  *  B,  is  equal  to 
(4.19).  i= 

It  follows  from  the  lemma  that  the  rank  uf  th*  system  (4.20)-(4.21 ) 
is  m  (the  system  is  triangular),  and  thus  v  may  take  any  non-negative 
arbitrary  value 

* 

We  shall  select  for  v  the  value  c$  where  s  is  again  the  project 
among  those  accepted  which  has  the  minimum  net  present  yalue  per  dollar 

invested.  This  choice  will  insure  that  all  accepted  projects  have  non- 

* 

negative  benefit,  that  is,  ct  -  v  >0.  Thus,  denoting  by  I  the  set  of 
rejected  projects  and  by  !  the  accepted  set,  we  have 

a 


c  >  0  ,  1  e  I 

5  “  a 


s  * 


1  r.  I 


(4.22) 
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Observe  that  it  may  still  be  possible  that  for  a  rejected  project 
ci  >  cg  implying  that  this  project  has  a  positive  benefit  (ct  -  c$  >  0). 

We  may  contrast  (4,22)  with  the  linear  programming  ^arc  studied  In 
Section  4.2  where  the  following  conditions  were  satisfied: 


c,  cs  >  0  ,  lei, 
S  '  5  0  •  1  £  <o 


(4.23) 


Thus  in  the  integer  programming  context,  as  Weingartner  remarks, 
rejected  projects  for  which  c^  -  c$  >0  are  essentially  taxed  or  penalized 
to  eliminate  any  profit  on  them,  thereby  preventing  their  acceptance. 

Finally  we  shall  reconcile  the  results  of  the  Integer  duality  theory 
developed  above  with  Weingartner's  ''alternate  dual  values"  approach  to 
establishing  shadow  prices  In  the  integer  capital  Investment  problem.  He 
assume  ([6],  pp.  103-106)  that  an  optimal  solution  x  ,  z  to  the  primal 
integer  program  Q  is  available  (l.e. ,  the  sets  I0  and  la  are  given).  He 
evaluates  the  "alternate  ;al  variables"  by  solving  a  linear  programming 
model  constructed  in  such  a  way  that  It  allows  negative  benefits  only  for 
rejected  projects.  The  model  he  proposes,  re-expressed  in  our  terminology 
and  notation,  is  is  follows: 


w  :  Minimize  2  5  Bv  «•  £  u.  x4 


'  *  ul  ?  c,  •  •  E  ‘4 

(4.24) 

»  •»,-  Y,  >V  1  £  I0 

(4-25) 

V.  u,.  y,  >  0 

where  the  benefit  (c^  -#)  is  given  by  >  0  for  accepted  projects  and 
by  (o,  -  Vj)  for  rejected  projects.  Under  the  assumption  that  a^  >  0  and 
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c1  >  0  and  that  Ia  t  $,  Welngartner's  probi  it  may  be  solved  by  Inspection 
as  warranted  by  the  following  theorem 

THEOREM  4  2  The  -  ,  u.,  v.  t^t-Xite  xn  jfitmal  iolwUon  to  mi 

ohZu  v  +  u  c  ,  .<.  e  T  and  j  -  u  -  y •  •  :  <  e  I„. 

'  J  L  L  &  >.  M.  V  0 

ft  k 

Proof:  For  any  v  >  0,  u,  and  y,  ,  t  £  IQ  exist  satisfying  (4.25)  as  a 
strict  equality  without  alte"ng  the  value  of  the  object! re  function.  Now 
consider  constraints  (4.24): 

*  ft 

a)  v  -  0  -j,  -  0,  V-  I  -  »s  not  possible  since  !  is  assumed 

lid  a 

non-empty. 

b)  if  v  -  0,  than  u.  >  0.  (If  u.  *  0  for  a  subset  of  I  ,  then 

'  1  ★  a 
c.  -  0,  or  else  the  solution  v  ,  u,  is  infeasible.)  Assume  also 

*  ft  * 

v*  +  o,  >  c  ,  i  e  I  .  Then  each  u?  may  oe  decreased  without 

1  1  a  1 

violating  the  constraints  (4.24)  but  with  a  decrease  in  the 

*  * 

value  of  the  objective  function.  Thus  v  ♦  u]  =  c^. 

*  *  ft 

c)  v  >  0,  u,  s  0  for  some  i  e  I  and  v.  >  0  for  the  remaining 

*  ft  ®  »  * 

i  e  I  .  Assume  also  v  >  c.  for  all  i  such  that  u.  *  0-  Then 

4  a  1  1 

v  may  be  decreased  without  violating  the  constraints,  causing 

*  * 

the  objective  function  to  decrease  Therefore  v  *  c,.  If  v 
*  *  *  1 
■  u.  >0  still  holds  for  i  such  that  u1  >  0,  then  may  be 

decreased  without  violating  the  constraint,  causing  the  objective 

ft  ft 

to  decrease  Thus  v  *  =  y  Q  =  D 

Furthermore ,  by  taking  the  dual  of  w  it  can  be  shown  that  the 

ft  ft 

optimal  solution  2  to  w  is  equal  to  the  assumed  known  value  z  of  the 
integer  problem  Q.  Thus  the  solution  of  w  is  reduced  to  solving  the 
system  of  equations : 
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*  »  *  * 

Bv  +  L  u4  x  *  i 

'«».  '  ' 


(4.26) 


V  ♦  u  »-  c 
1  1 


,  tel. 


*  «  * 


v  •  u,  •  K,  •  C,  .  1  c  I0 


*  *  * 

v  .  V  Yt  a  0 


This  system  is  soiled  as  follows:  If  I  x  B,  then  ; 4 . 26 )  is 

<eIa 

*  * 

redundant  ;  th^n  by  selecting  v  *  cs  arbitrarily  with  c$  being  the 


smallest  ct ,  1  -  i  ,  the  solution  is 


v  “  c. 


U1  *  ci  -  cs  >  0 


i  e  I. 


(4.27) 


U1  ~  "  ci  '  cs  J  0  *  1  e  \ 


Comparing  now  the  results  of  (4.22),  obtained  by  direct  exploitation 
of  discrete  duality  theory,  with  (4  27),  derived  from  Weingertner's  in¬ 
tuitively  constructed  model  w,  we  observe  that  they  are  the  s»?e,  thus 
establishing  the  equivalence  of  both  approaches  in  determining  a  system 
of  shadow  prices  for  the  capital  investment  problem  under  consideration. 


If  I 


*  * 

x4  <  8  _  solution  Is  v 


0,  o. 


i  c  I , 


i  e  I. 
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4.4  SOLUTION  OF  THE  KNAPSACK  PROBLEM 

The  branch  and  bound  algorithm  of  Section  3.6  may  be  directly  applied 
to  solve  the  knapsack  problem  as  formulated  in  problem  Q.  However,  Its 
application  to  this  single  period  Investment  problem  is  totally  deter¬ 
ministic  in  the  sense  that  when  a  branching  operation  is  about  to  take 

place  from  a  bounded  node,  the  solution  of  the  auxiliary  problem  for  that 

* 

node  contains  only  one  varialbe  with  0  <  x^  <  a^.  Hence  this  particular 
variable  must  be  a  facfutioni  be  fixed  to  its  only  possible  values  xi  *  0 
and  Xj  *  a;  in  order  to  continue  the  execution  of  the  algorithm. 

To  illustrate  its  application  and  to  compare  it  with  the  alternative 
algorithm  of  Section  4.5,  consider  the  following  problem. 

EXAMPLE 

Assume  that  the  ten  projects  of  the  example  in  Section  3.7  are 
considered  for  investment  with  the  same  payoffs  and  with  the  same  total 
outlays  required  except  that  these  outlays  nst  be  invested  in  a  single 
time  period.  Table  4-1  shows  the  pertinent  data.  The  projects  are  again 
oro^red  in  decreasing  values  of  their  c..  The  budgetary  ceiling  is  B  =  70. 

To  initialize  the  problem  and  thus  generate  the  root-node  1,  problem 
Q  is  solved  ignoring  constraints  (4.4).  We  observe  in  passing  that  the 
solution  proposed  by  Lorie  and  Savage  corresponds  to  the  solution  of  the 
auxiliary  problem  associated  with  the  root  of  the  solution  tree.  This 

solution  is  z  =  79.15  and  x  *  [8,  12,  13,  12.  15,  10,  0,  0,  0,  0], 

* 

where  =  10  is  the  enly  variable  root  zero  or  at  upper  bound;  thus 


— 

Project 

No 

ai 

il 

ci 

1 

15 

8 

1.875 

c 

17 

12 

1.417 

3 

15 

13 

1.154 

4 

12 

12 

1.000 

5 

14 

15 

0.933 

Project 

No 

f1 

ai 

fi 

ai 

6 

40 

65 

0.615 

7 

12 

21 

0.57’ 

8 

17 

61 

0.279 

9 

,  « 

«  K 

52 

0.269 

10 

10 

39 

0.256 j 

TABLE  4.1 
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the  branching  will  take  place  from  node  1  by  fixing  x^  *  0  and  Xg  =  65. 
The  rounding  solution  associated  with  the  node  is  x  :  [8,  12,  13t  12,  15, 
0,  0,  0,  0,  0]  with  i  3  73.  The  optimal  solutions  of  nodes  11  and  21  are 
obtained  after  11  iterations  of  the  algorithm.  The  pertinent  information 
at  each  Iteration  >s  recorded  in  Table  4-2  and  the  solution  tree  in  Fig. 
4-2.  We  observe  that  since  the  rounding  operation  at  node  1  produces  a 
solution  which  turns  out  to  be  optimal,  the  t^nming  of  the  solution  tree 
by  use  of  the  rejection  operation  is  very  po«e'fui  tor  this  particular 
example. 


STEP 

i 

Feasible  nodes 

F(D 

Excluded 

nodes 

Rejected 

nodes 

R(D 

r  ■ — -i 

Lower 

bound 

Li 

Upper 

bound 

Ui 

1 

* 

1 

73 

79 . 1 5+ 

2 

* 

3 

2 

73 

78.71 

* 

3 

4  ,  5 

73 

75.79 

4 

* 

4  .  7 

6 

73 

75.69 

J 

* 

4  ,  9 

8 

73 

75.56 

6 

* 

4  ,  11 

10 

73 

74.73+ 

7 

11  ,  12* 

13 

73 

74 

8 

11  ,  15* 

14 

73 

73.28 

9 

11  ,  17* 

16 

73 

73.27 

10 

11  ,  19* 

18 

73 

?6 

11 

11  ,  21 

20 

73 

73 

*  :  bounded  node 

t  :  search  on  a  new  branch  of  the  solution  tree 


TABLE  4-2 


F.-uhi  Table  4-2  we  observe  that  information  for  no  more  than  two 
nodes  need  be  stored  at  any  step  of  the  solution  process.  The  number  of 
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nodes  remaining  in  the  feasible  set  at  the  last  iteration,  F(11),  Indicate 
the  number  of  optimal  solutions.  For  this  example  two  strategies,  both 
not  exhausting  the  budget,  are  available-  a)  accept  projects  1,  2,  3,  4 
and  5;  b)  accept  projects  1,  2,  3,  5,  7.  Note  that  the  optimal  solution 
has  a  total  utility  of  73,  as  opposed  to  70  in  the  example  of  Section  3.7 
where  the  buaget  ceil  mgs  were  given  m  two  time  periods 


Number  of 


Step  i 


FIG.  4-1 

In  Fig.  4-1,  the  difierence  between  the  lines  (a)  and  (b)  is  fixed 
since  the  branching  operation  is  deterministic.  Line  (c)  indicates  the 
"umber  of  terminal  nodes  to  be  stored  a'  each  iteration  The  difference 
between  lines  (b)  and  (c)  is  the  result  of  the  reaction  operation  of  the 
algorithm.  Note  that  the  rate  of  increase  of  terminal  nodes  with  number 
of  iterations  performed  is  drastically  reduced  by  the  rejection  rule  in 
this  case 

According  to  the  theory  of  Section  4.3=  the  imputed  dual  prices 
*  * 
would  be  v  •  0,  since  the  budget  is  a  free  good  and  s,  *  -  c-. 
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4.5  A  PARAMETRIC  BRANCH  AND  BOUND  ALGORITHM  FOR  THE  KNAPSACK  PROBLEM 

This  section  is  devoted  to  the  derivation  of  an  alternative  algorithm 
for  the  solution  of  the  knapsack  problem.  This  method,,  as  will  be  shown, 
is  a  special  case  of  the  branch  and  bound  a'gorUhm  of  Chapter  II,  with  a 
more  elaborate  branching  mechanism.  The  algorithm  has  certain  features 
similar  to  the  so-called  4jtnpt%.zvt  methods  [7],  [8],  [9].  The 

most  important  character  is t’ c  of  the  method  studied  here  is  that  it 
possesses  the  special  property  that  the  final  tree  configuration  ontains 
the  optimal  solutions  for  a!'  similar  knapsack  problems  with  larger  bud¬ 
gets  than  the  one  utilized  to  generate  the  tree.  The  branching  part  of 
the  algorithm  is  similar  to  the  one  employed  by  Ichbiah  [3]  on  a  network 
connectivity  analysis  problem  which  guarantees  the  parametric  properties 
of  the  resulting  algorithm. 

Let  us  rewrite  our  bivalent  linear  programming  formulation  of  the 
knapsack  problem  where  optimal  x°  and  z°  are  sought  as  to 


Q  :  Maximize 

Subject  to 


2(B)  - 

m 

E  c 
i=l  1 

X 

l 

m 

l  X. 

<  B 

1  =  1  1 

0  <  x. 

<  a. 

,  XL 

-  i 

i 

1 

*1  *° 

or  a1 

*  *1 

Here  z°(S)  is  the  kacLpiack  ^unctco1!  corresponding  to  a  specific 
value  B  of  the  budget.  Let  s.s  define  the  sets  s^ ,  T1  and  as  follows: 

S1  ‘  /  Q  I  V  a>!  (4-28) 

Ti  1  lx  /  ?  x.  <  B;  x.  *  0  or  a.}  (4.29) 

1  1=1  1  ’  1  1 

si,  .  s,ni|  ■  r, 


(4.30) 
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The  set  S-j  is  closed  and  bounded  in  Em;  T^  is  a  nonempty  and  finite 
set  for  B  >  0  and  thus  is  finite. 

Stanching  OpeAaticm.  Given  a  certain  node  l  of  the  solution  tree  with 
associated  sets  and  s^,  the  branching  is  defined  by  the  intersection  of 
the  sets 


'l,r  =  {x1  1  \  =  H} 

(4.31) 

'i,» i *  '"i  >  xk  - ot 

(4.32) 

for  a  given  i  =  k.  The  sets  thus  defined  satisfy  the  sufficient  conditions 
of  Theorem  2.1,  as  may  be  easily  verified.  Also,  since  at  each  branching 
operation  one  variable  is  set  to  its  only  possible  values  and  the  number 
of  variables  is  finite,  the  complete  enumeration  of  solutions  and  thus  the 
finiteness  of  the  algorithm  is  guaranteed. 


hi.xJUU.ojui  Pfioblm.  Associated  with  a  newly-generated  node  l  of  the  solution 

tree,  a  continuous  problem  derived  from  Q  must  be  solved.  Denoting  by 

I.  I.»  and  I  the  sets  of  variables  fixed  at  a  zero  level,  fixed  at  the 
0  a 

upperbound,  and  free,  problem  A^  takes  the  trivial  form: 

A,  :  Maximize  z[l)  *  ?  c.  x. 

1  1»1  1  1 

Subject  to  0  <  X|  s  ij  ,  1  e  I 


Since  we  are  maximizing  over  the  set  of  free  variables,  bounded  from 
above  and  since  c^  >  0  Is  assumed,  this  linear  program  has  as  optimal 
solution: 
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•  1f  r  e  Jo 
,  <  f  f  c  T 


(4.33) 


Note  that  A^  possesses  always  an  optimal  solution  and  therefore  the 
exclusion  operation,  as  denned  in  Chapter  II,  will  never  be  applicable 
to  this  problem. 


Tiwu.na.ti.on  R aiz  Rejection  operations  wiH  not  be  used  in  this  approach, 

hence  the  termination  'j'e  becomes  the  foM owing  as  nay  be  easily  verified: 

* 

terminate  whenever  the  solution  *.  to  the  auxiliary  problem  of  the  current 
bounded  node  is  feasible,  (i.e. ,  <  e.  fi, ). 


LEMMA  4,2  A  autu Uon  x  to  4^  te  (,iabi.bti  £01  Q  ct  atbo  bxtlb^ite 


L  A.  • 

X*1  c 


*  * 

Proof:  If  x  is  optimal  for  A*  ,  then  from  (4.33)  x.  *  0  or  a. ,  -V-,  and 
m  *  7  v  *  '  ,  .  '  *  * 

If  alto  I  x  <  B,  then  from  (4  29)  x  e  T,  and  from  (4.30),  x  e  Slr 
1«1  1  1  1 


COROLLARY  4.1  I £  x  u>  op  tonal  501  A^  u,\th  £  x  >  8,  thi  bit  S ^ 

ceIa 

abboclatid  uuAk  nodi  l  dote  not  cottcurt  a  ^iatxbli  botutxcn  to  Q. 

it 

From  the  above  corollary,  a  node  r  of  the  tree  for  which  £  ^  >  B 
may  be  excluded.  Observe  however  that  for  a  larger  value  of  1e*a 
the  budget  the  condition  of  the  corollary  may  not  be  satisfied  and  the 
branch  would  not  be  deleted  at  that  step. 

Up  to  here  the  development  parallels  the  additive  algorithm  of  Bales 
as  Implemented  by  Geoffrion  [8],  with  Corollary  31  providing  tne  first 
rejection  rule  0 *  Balas.  From  the  above  discussion  it  has  been  shown 
that  Bales'  enumerative  algorithm  may  be  interpreted  as  a  branch  and 
bound  algorithm  of  the  Land  and  Oolg  type 
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However,  from  here  on  our  approach  diverges  from  [8]  in  order  to 
provide  the  parametric  feature  of  the  algorithm. 

fixzd.  aiii.d'-j4  When  a  oranch’ng  operation  is  to  be  performed 

based  on  the  currently  oounded  node  l,  one  of  the  free  variables  must  be 

selected  to  be  fixed  at  its  two  possible  values.  We  shall  select  the 

* 

following  criterion  that  will  render  "deterministic"  the  generation  of 
the  branch  and  bound  tree:  Select  fo<  branching  *s  s  c  I;  such  that 

f  1  mm  [f  ]  (4.34) 

it! 


If  there  is  a  tie,  select  the  variable  with  greatest  index.  With 
this  selection  rule  we  create  the  directed  arcs  (l,V)  and  (£,r).  For  the 
new  node  l1,  -  a$,  (project  s  is  accepted),  the  optimal  solution  to  the 

auxiliary  problem  A^'  would  be  the  same  as  the  solution  to  A^;  furthermore, 
that  solution  is  not  feas'ble  for  problem  Q*  Thus  node  V  will  be  denoted 
a  paeucto-nodc  and  no  extra  computation  will  be  necessary  whenever  such  a 
node  is  generated.  As  for  node  r,  the  solution  to  the  auxiliary  problem 
Ar  has  a  value  z  (r)  ^  z  (£)  where  [z  [L)  -  z  (r)]  is  the  smallest  de¬ 
crease  possible  in  the  objective  function  since,  by  (4.34),  the  variable 
fixed  to  zero  is  the  one  that  has  the  minimum  payoff  of  the  set  of  free 
variables 

Whenever  a  pseudo-node  V  is  generated  from  bounded  node  l,  then 

★  * 

z  [V  )  *  z  (t)  and  the  next  bounded  node  of  the  solution  tree  would  ob¬ 
viously  be  1‘  Accordingly,  whenever  benching  takes  olace  from  l  to  l' , 
an  extra  branch  fro.--  ?'  wi'l  also  take  place  This  additional  branch, 
performed  by  fixing  to  zero  a  new  variable  selected  according  to  (4.34) 
will  permit  us  to  "look-ahead"  on  the  solution  tree  We  remark  that  the 
branch  from  t'  fixing  the  selected  variable  to  its  upper  bound  remains  to 
be  performed  and  the  algorithm  must  provide  for  Its  convenient  generation. 


In  the  sense  that  an y  run  of  the  problem  would  produce  the  same  tree. 
*  Iv  it  was,  f being  the  bounded  node,  it  would  be  an  optima1  solution. 
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4.6  STATEMENT  OF  THE  ALGORITHM 

Having  satisfied  the  assunipn  Dns  of  the  a'gonthm  of  Chapter  II,  and 
having  indicated  the  branching  mechanism  to  be  employed,  we  proceed  to 
state  the  px'uvnzt'uc  bn.xnih  xni  boui id  algorithm. 

STEP  1.  Set  i  -  ■  Gene* ate  node  >  by  so'vng  A,  according  to  (4.34). 

«  «  *  in  * 

Let  2  ,  «  be  the  optima*  so'ution  and  det’ne  B  -  l  x..  If 

i-I  1 

* 

B  <  B,  stop;  tne  so'ution  is  optima'  Otherwise,  bound  node  1 
* 

with  'J.  •  2  Set  i  =  i  ♦  !  and  go  to  step  i 

STEP  i 

A. 1,  BRANCH-  Branch  from  bounded  node  l.  Select  the  variable  xg 
according  to  (4  34) .  Create  pseudo-node  V  and  node  r,  and  directed 
arcs  U,C)  and  [t,r)  Solve  problem  Ar  with  =  0.  Set  x$  =  a$  for 
pseudo-node  V  , 

BRANCH  AHEAD  Branch  from  pseudo  node  V  .  Select  a  new 
variable  x..  according  to  (4.34) „  Create  node  r  +  1  and  directed  arc 
(£' ,  r  +  1)  Solve  problem  A  ,  with  *t  =  0.  If  the  unique  predecessor 
lQ  of  bounded  node  l  is  a  pseudo-node,  go  to  A  2;  otherwise  go  to  B. 

A.  2  BRANCH  Branch  from  pseudo-node  i  Let  x  be  the  variable 

r 

fixed  to  zero  associated  with  the  arc  U0»t)  Create  pseudo-node  l'Q  and 
directed  arc  UQ,tg)  Set  *p  -  ap  for  pseudo-node  i'0 

BRANCH  AHEAD  Branch  from  pseudo-node  t The  variable  to  be 

o 

fixed  is  Create  node  r  ♦  2  and  directed  arc  U0,  r  ♦  2)  Solve 

problem  A  0  with  x.  1  0  Go  to  B 
r  r+c  t 

B.  BOUND  Select  node-  L  such  that  i  (t)  -  max  iz  { k ) )  for  current 

terminal  nodes  It  B  <  B  for  node  i,  stop;  me  solution  associated  with 

* 

node  l  is  optimal  Otherwise,  upperbound  node  £  with  U  *  z  (£),  set 
i  =  i  +  1  and  go  to  step  i 
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The  solution  tree  generated  for  a  knapsack  function  with  B  *  BQ 
contains  the  optima'  solutions  (projects  accepted  as  well  as  maximum  pay¬ 
off)  for  all  knapsack  functions  with  B  >  B^,  as  ;s  shown  below  In  this 
sense,  the  algorithm  is  parametric.  By  netting  BQ  -  G  and  applying  the 
algorithm,  a  table  may  be  instructed  with  the  optima’  solutions  to  problem 
Q  for  any  non-negative  <a  ue  ot  the  buageta'y  ceiling. 

LEMMA  4,3,  ut  iJ  8  xni  .J  8  bt  iZ'Up-.*..?  5-xac.t  m.-  tu)o  valuer 
nj  the  budget,  8  l  6  riun  8  l  8, 

Proof:  Assume  z°iBj  2J;80/  and  'el  1°  the  *->pt *  na  1  solut'on  for  the 
budget  Bq  Since  B  _  Bq,  then  x°  is  also  leasib’e  for  the  budget  value 
B,  lienee  2(B)  ;  20(BQ)  for  *°,  a  contradiction.  Therefore, 
z°(B)  >  z°  (Bq) 

THEOREM  4,3  let  T  [N  ,  A  ]  be  the  j-<.na£  ta«.e  genemtca  by  determining 

0  j  0 

the  optimal  solution  to  Q  joa  8  ■  B  Then  jit  any  8  >  8  theie  exists 

a  node  beN  such  that  the  solution  to  the  auxtfuUu  problem  A.,  aai/ 

*  •  o  j 

z  ,  x  ,  conat-ctutea  an  optimal  solution  jo  i  the  eittea  ponding  knapsack 
problem. 

Proof:  Given  B  >  Bq,  the  application  o'  the  algorithm  would  generate  a 
tree  T  -  [N,  A]  for  which  at  'east  one  node  k  i  N  corresponds  to  the 
optimal  solution  The  tree  T  is  a  yibgxaph  of  Tq,  that  is,  nCnq  and 
for  all  (i,  j)  c  A,  a'so  li,  j)  t  Aq  In  effect,  since  the  algorithm  is 
deterministic,  the  nodes  of  Tq  and  T,  gene'ated  m  the  same  order,  will 
have  the  same  ussoaated  solutions  to  the  auxiliary  problems  A^  It 
remains  to  be  proved  that  when  generating  T,  the  optimal  node  k  corres¬ 
ponds  to  a  certain  node  of  the  generated  portion  jr  I  .  Assume  the 

*  0  o 

contrary,  k  t  N  ;  then  the  solution  to  A  gives  z  (B)  <  z  (B  ),  since  ary 

O  ^  q 

additional  branching  from  Tq  reduce*  the  upperbound  value  i  (9  ).  From 
Lemma  4  3  this  is  a  contradiction,  and  the  solution  for  B  >  Bq  is  not  in 
the  ungenerated  portion  of  Tq,  hen^e  k  r  Nq  it  also  fellows  that  T  is 
a  subgraph  of  Tq 
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We  now  give  the  following  rule  for  retrieving  the  optimal  node  k. 

The  set  of  bounded  nodes  at  each  iteration  of  the  algorithm  for  B  ■  BQ, 

form  a  sequence  [1,  t ty  ,£n]  where  £( ,  1  e  Nfl  is  the  bounded  node 

used  for  branching  at  iteration  1  and  l  is  the  optimal  node  for  B  *  B  . 

n  *  *  o 

Associated  with  each  t,  the'-e  are  two  numbers,  i  (£,)  and  B  (£.),  Due  to 

1  *  I  * 

the  way  the  branch  and  bound  algorithm  has  been  developed,  the  z  [l 

values  constitute  a  non- increasing  sequence  Therefore,  for  a  given  value 

* 

B  >  B  it  suffices  to  retrieve  the  first  node  for  which  B  <  B, 

-  o 

4.7  SOlUT'ON  Of  AN  ExAMPtE  hROBlEM 

As  an  example  consider  the  following  problem  involving  five  projects 
with  payoffs  and  capital  outlays  as  indicated  in  Table  4-3.  Table  4-4 
contains  the  necessary  information  for  each  iteration  of  the  method.  The 
budgetary  ceiling  considered  is  B  =  10. 


- - —  — 

Project 

No. 

f1 

di 

1 

6 

3 

2 

4 

5 

3 

3 

6 

4 

0 

fc» 

4 

5 

i 

2 

TABLF  4-3 

Note  that  a  feasible  solution  is  obtained  at  iteration  4  when  node 
5  is  generated,  but  not  until  iteration  7  can  it  be  bounded.  Note  also 

that  at  iteration  7  (when  branching  from  node  13),  all  variables  for  the 

■* 

pseudo-node  13'  are  fixed,  its  B  value  is  greater  than  the  budget,  and 
therefore  it  may  be  eliminated  from  further  consideration.  The  same  may 
be  said  of  node  15. 

The  optimal  solution  for  8  ■  10  is  then  obtamed  from  node  5. 
accept  projects  1,  2  and  5  with  ;°(I0)  5  !’ 
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From  Table  4-4  any  optima*  solution  fo*  B  >  10  may  be  obtained.  It 

suffices  to  search  down  the  column  of  B  and  'ead  off  the  solution  from 

* 

the  row  for  which  the  first  B  less  than  o t  equal  to  the  budget  of 


Step 

1 

ierminal  nodes 

B  for 

bounded 

node 

_ 

Uppe. oound 
Ut 

* 

1 

16 

2*,  3 

15 

3*.  4  ,  5 

14 

4*.  5  ,  6,  7  ,  8 

14 

13 

5  ,  6  ,  7,  8*.  9,  10 

14 

13 

* 

5  ,  6  ,  7.  9  ,  10,  11,  12,  13 

15 

12 

7 

5*.  6  ,  7,  9  .  10.  11  ,  12,  14,  15 

12 

12 

8 

6*.  7,  9,10,11,12,14,  15,  16,  17 

10 

11 

bounded  node 


TABLE  4-4 


interest  is  encountered  For  8  *  14  the  fourth  iteration  provides  an 
optimum,  with  z°(14)  *  13  associated  with  the  bounded  node  4,  namely: 
accept  projects  1,  2,  and  3  An  alternative  optima  is  given  by  node  8: 
accept  projects  1,  2,  4  and  5 

In  Fig,  4-4,  the  op  .  nai  solutions  for  B  >  10  are  indicated 
graphically  Fig  4-3  contains  the  required  solution  tree. 
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z°(B) 


16 

11 


8 


FIG.  4-4  Knapsack  Function  Values 

4.8  SUMMARY 

In  this  chapter  we  have  addressed  ourselves  to  the  solution  of  the 
Integer  programming  problem  known  as  the  knapsack  problem.  We  have  ob¬ 
tained  the  solutions  to  the  dual  problems,  both  in  linear  end  the 
discrete  case,  and  have  discussed  the  natural  econom'c  interpretation 
that  may  be  drawn  from  such  solutions.  We  have  used  discrete  programming 
duality  theory  to  justify  Welngartncrs  approach  t.i  calculating  dual 
prices  on  the  primal  resources 

The  branch  and  bound  algorithm  developed  for  the  multidimensional 
knapsack  problem  in  Chapter  III  has  been  applied  to  the  single  period 
capital  rationing  problem  with  the  consequent  simplifications  indicated. 
F-.nally,  a  parametric  branch  and  bound  algorithm  has  been  derived  which 
provides  for  sensitivity  analysis  studies  of  the  optimal  solution  for 
variations  of  the  budgetary  ceiling  within  a  certain  specified  range. 
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In  the  following  chapter, 
be  provided  for  the  problem  of 
where  the  investment  decisions 


formulation  and  solution  techniques  will 
capital  allocation  to  independent  projects, 
may  be  deferred  to  later  periods. 
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CHAPTER  V 


MULTISTAGE  RESOURCE  ALLOCATION  PROBLEMS 


5.1  INTRODUCTION 

In  the  preceding  chapter  we  studieu  the  proDlem  of  optimally 
allocating  funds  among  competing  alternatives,  each  requiring  investment 
in  a  number  of  time  periods  and  with  a  fixed  horizon,  subject  to  budget¬ 
ary  constraints  in  each  period.  An  effective  solution  technique  was 
developed.  The  purpose  of  this  chapter  is  to  study  the  problem  of  capital 
allocation  among  independent  projects  that  arises  in  various  planning 
contexts,  where  the  projects  require  fixed  outlays  in  one  time  period,  but 
the  decision  of  accepting  them  may  be  deferred  to  a  later  period.  An  ex¬ 
pected  net  benefit  (financial  and  social)  is  assumed  to  be  known  for  each 
project,  and  the  figure  of  merit  adopted  is  to  maximize  the  total  benefit. 

We  begin  by  formulating  the  multistage  capital  allocation  problem  as 
a  (0-1)  integer  program.  A  special  case  is  then  considered  where  capital 
outlays  for  all  projects  do  not  vary  over  time  and  any  infationary  effects 
are  taken  into  consideration  by  modifying  the  budget  ceilings  accordingly. 
The  problem  is  referred  to  as  the  muIU-knaptack  protein.  A  suitable 
transformation  is  performed  to  obtain  an  equivalent  model  which  is  inter¬ 
preted  as  an  analysis-syntheses  problem  on  a  bipartite  network.  The 
general  branch  and  bound  algorithm  of  Chapter  II  is  then  adapted  to 
provide  a  convenient  solution  methid. 

Finally,  a  second  formulation  for  the  multi-knapsack  problem  is 
derived  for  which  a  branch  and  bound  algorithm  is  proposed.  In  this  case, 
the  solution  of  the  auxiliary  problem  associated  with  each  node  of  the 
solution  tree  may  be  obtained  by  inspection 
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5.2  MULTISTAGE  CAPITAL  ALLOCATION  MODELS 

Consider  a  certain  government  agency  confronted  with  the  problem  of 
allocating  a  multlstaged  budget  with  ceilings  >  0,  j=l,,..,n,  among  a 
set  of  m  Independent  projects  each  requiring  a  unique  capital  outlay. 

Let  f,j  be  the  expected  payoff,  determined  from  a  linear  utility  function, 
of  Investing  in  project  i  at  time  period  j.  Further,  let  a.  .  >0  be  the 

*  J 

capital  outlay  required  for  project  i  if  selected  for  investment  at  time 
period  j,  and  y^  the  associated  decision  variable  that  may  take  on  the 
values  0  or  1  depending  on  whether  project  i  is  rejected  or  accepted  for 
investment  in  period  j.  Then  the  problem  of  selecting  a  set  of  projects 
for  investment  so  that  the  total  utility  over  time  is  maximized,  while 


satisfying  the  funding  dependencies  represented  by  the  budgetary  ceilings 
may  be  formulated  as 

:  Maximize 

z  = 

2  S 

i=1  j=l 

fu 

Subject  to 

m 

l 

1*1 

3ij  yij 

sB.  ,  j=l ... .  ,m 

J 

(5.1) 

n 

l 

j*1 

yij^ 

,  i=l . n 

(5.2) 

yij 

>  0 

(5.3) 

yij 

integer 

1 

(5.4) 

where  constraints  (5.1)  express  the  budget  limitations;  and  where 
constraints  (5.2)  serve  the  double  task  of  guaranteeing  that  project  i, 
if  accepted  at  any  one  period,  incurs  a  unique  capital  outlay  and,  con¬ 
currently  with  (5.3)  and  (5 A),  that  the  variable  y^  may  only  take  the 
values  0  or  1. 

Problem  P|  is  an  all-integer  linear  programming  problem  with  (m+n) 
constraints  and  ?:ixn  integer  variables  for  which  a  feasible  solution  and 
a  lower  bound  are  immediately  available,  corresponding  to  the  itaXui  quo 
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or  reject-all -projects  policy..  Observe  also  that  by  collapsing  the  number 
of  time  periods  to  one,  the  index  j  may  be  dropped  and  thus  problem 
becomes,  as  expected,  the  (0-v)  knapsack  problem  studied  in  Chapter  IV. 

The  so 1  cit ion  to  problem  P j  may  be  attained  by  direct  application  of 
the  branch  and  bound  algorithm  of  Chapter  il,  In  this  case,  when  constraint 
(5.4)  is  relaxed  the  problem  becomes  the  well-known  teAghXe.d  di&tAibution 
problem,  sometimes  called  the  genzau LLzv.d  tAutupjAtaCcon  pKoblm.  There¬ 
fore  the  pnmai  method  of  Dantzig  [l]  or  the  dual  method  of  Balas  [2] 
could  in  principle  be  used  as  subalgor : thms  for  solution  of  the  auxiliary 
problems  associated  with  each  node  of  the  branch  and  bound  tree. 

We  shall,  however,  address  ourselves  to  the  special  case  of  P^  in 
which  the  capita1  outlay  for  each  project  remains  the  same  regardless  of 
the  period  cnosen  for  investment;  this  amounts  to  assuming  non-inflationary 
costs  throughout  the  horizon  of  interest.  The  problem  thus  obtained  has 
wide  application  to  various  resource  allocation  problems. 

5.3  THE  MULTI -KNAPSACK  PROBLEM.  AN  EQUIVALENT  MODEL 

Problem  P1  with  the  additional  condition  that  the  a^  are  the  same 
for  all  j,  which  we  shall  call  the  multi- knapiack  problem,  is  a  generali¬ 
zation  of  Dantzig's  knapsack  problem.  It  can  be  expressed  as  follows: 
determine  the  optimum  packing  of  a  set  of  m  articles  into  a  set  of  n 

knapsacks,  given  the  desirability  f.  of  each  item  for  each  knapsack,  the 

1 J 

weight  ai  of  each  item,  and  the  maximum  weight  B  •  that  each  knapsack  is 
allowed  to  carry 

This  problem  is  of  special  importance  in  the  operation  of  transpor¬ 
tation  terminals,  where  optimal  cargo  loading  into  vehicles  of  varying 
capacities  is  desired.  Also,  it  arises  in  a  computer  environment;  where 
programs  or  files  of  a  given  s i Zvi  are  competing  for  non-connected  fixed 
size  data  storage  pools.  These  a 'e  but  two  examples  of  optimization 
problems  that  can  be  modelled  as  multi-knapsack  problems. 

We  shall  study  this  problem  in  te-mis  of  an  alternative  model 
equivalent  to  problem  P^.  Witc.  this  equivalent  model,  the  problem  will 
be  interpreted  as  a  network  analysis-synthesis  problem  which  resembles 
the  plant  location  problem  with  fixed  charges  on  links  with  positive  flows. 

A  branch  and  bound  algorithm  tor  the  solution  of  this  problem  will  also 
be  developed. 
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Consider  problem  P,,  w*tn  a  ,  'ep  iced  by  a,  a*  discussed  above. 

1  ^  I 

Replacing  the  set  of  va^aoles  oy  the  new  vav?ao-es  ^  y^, 

letting  c , •  1  j .  we  obta’n  the  fo*' owing  eqj* valent  program: 

1  w  a 


and 


„  „  .  01  n 

P  .  Max' m" ie  i •  I  t 


X 

J  J 


xij  -  ej  »  j  =  K-  - .  ,n 

(5.53 

n 

l  V  <  a,  ,  i-i - - m 

j-i  lj 

(5.6) 

(5.7) 

Problem  P  is  a  discrete  bivalent  linear  programming  problem  where  the 
represent  capital  outlays  to  be  determined  and  the  c..  represent  the 

'  J 

utility  of  project  i  at  time  rePod  j,  per  un-t  of  outlay  required. 

Let  us  interpret  the  set  N,  •*  [i  /  i  -  ’  as  a  set  of  "origin" 

nodes,  Ng  =  [j  /  j  =  l,...,nj  as  a  set  of  "destination"  nodes,  and  the 
set  A  of  ordered  pairs  is  a-cs  joining  nodes  i  and  j.  Furthermore, 

let  a,.  be  the  "demand"  input  of  node  ’  e  N,  and  B  the  "demand"  or 

J 

output  of  node  j  e  N^,  ano  interpret  xi .  as  Pow  on  the  arc  (i,j).  We  may 
then  associate  a  bipartite  network  G  *  f'l,,  Ng  A]  to  problem  P;  or  better 
still  an  equivalent  network  with  a  single  source  and  a  single  sink.'  This 
latter  step  may  be  achieved  by  adding  artificial  nodes  s  and  t,  and 
artificial  arcs  (s  ,0 ,  ¥■  >  e  N,  and  (j  ,t' ,  V-  j  e  Ng.  with  the  following 
associated  values:  c$1  1  0,  r  a(  and  0,  u  r  B^;  where  u^ 

and  u  .  denote  respectively  the  upper  bounds  on  the  arcs  (s,i)  and{j,t  ). 

J  W 

The  associated  netwo'k  is  shown  in  F ■ g .  5-1.  The  first  number  on  each 
arc  represents  cost  and  the  second  -epresents  arc  capacity 
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FiG.  5-1 

The  multi -knapsack  problem  therefore  may  be  expressed  in  terms  of 
network  flow  theory  as  an  MCLlytii-  -tynthi. problem:  find  the  maximum 
flow  from  s  to  t  that  maximizes  cost  on  the  network  of  Figure  4-1,  subject 
to  the  restriction  that  arcs  0 , j ) ,  i  e  N1 ,  j  e  are  either  not  used  or 
saturated;  and  find  as  well  its  distribution  pattern  Note  that  since  the 
upper  bounds  on  arcs  (s,i)  are  a)t  if  project  r  is  accepted  only  one  arc 
(r,j)  will  be  activated. 

The  optimal  solution  determines  which  projects  will  be  accepted  for 
investment  (not  al i  arcs  (s,i)  need  be  saturated);  and  it  determines  to 
which  destination  node  they  wi i ’  be  assigned,  thus  completing  the-synthesis 
portion  of  the  problem 

Observe  that  problem  P  differs  from  a  standard  transportation  problem 
with  surplus  and  deficit  in  that  each  origin,  w  j;,ed  at  all  for  shipping, 
must  supply  a  single  destination  node.  (It  also  differs  in  tnat  P  is  a 
maximization  problem)  If,  however,  the  constraint  (57)  is  relaxed  so 
that  the  x?j  are  simply  restricted  to  be  non-negative,  the  resulting  pro¬ 
gram  indeed  corresponds  to  a  transportation  problem  with  surplus  and 
deficit  This  fact  will  be  employed  in  proposing  a  subalgorithm  for 
solution  of  the  auxiliary  problem  during  our  development  of  the  branch  and 
bound  solution  method  in  Section  5  4 

The  model  as  presented  in  formulation  P  may  be  extended  to  consider 
more  flexible  cases  which  might  add  relevance  to  the  problem  or  be  adapted 
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to  a  more  realistic  situation.  When  budget  deferrals  are  allowed,  thus 
transferring  unused  capital  to  a  later  period,  the  P  formulation  must  be 
modified,  as  Indicated  below,  to  account  for  such  flexibility.  Let  be 
the  unused  budget,  If  any,  at  time  period  j.  Then  It  suffices  to  modify 
the  constraints  (5.5),  replacing  them  instead  with  the  following  one: 


m 

xu  ’  sj  ■  5j-i  *  Bj  •  ^ . " 


(5.8) 


where  sQ  =  0.  We  remark  that  the  coefficient  of  s^  is  one,  hence  no 
present  worth  factor  has  been  considered  in  (5,8)  and  thus  the  slacks 
simply  represent  idle  costs.  Other  generalizations  may  be  made,  such  as 
lending  and  borrowing  in  the  capital  market.  (See  for  example,  [3], 
Chapters  8  and  9.) 

The  network  representation  'dr  problem  P  with  (5.8)  instead  of  (5.5) 
would  correspond  to  the  one  of  Fig.  5-1  with  additional  directed  arcs: 

(n,  n-1),...  ,(2,1).  These  arcs  are  uncapacitated  (unless  deferred  ex¬ 
penditure  is  specifically  bounded)  and  have  zero  costs. 


5.4  DEVELOPMENT  OF  A  SOLUTION  METHOD 


We  shall  derive  a  branch  and  bound  method  for  the  solution  of 
problem  P  by  utilizing  the  same  terminology  and  notation  employed  in 
Chapter  II.  We  begin  by  defining  the  sets  T^  and  ^  as  follows: 


S 


1 


,  m  n 

U  /  l  x . .  <  B.,  z 
~  t=l  1J  J  j*l 


xij  £  xij  2  °-  *<'••!» 


=  (*  /  j  =  0  or  ai } 


ft,  =  S]  O  T] 


We  observe  that  S1  is  a  closed  and  bounded  convex  set,  since  it  Is 
defined  as  the  intersection  of  a  finite  number  of  closed  convex  half 
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spaces;  furthermore,  ?"y  vector  x  with  x,4  <  0,  ¥  1,j  constitutes  a  lower 

—  it 

bound,  and,  for  examp.;,  x^witn  constitutes  an  upper  bound  .  Also, 

T-j  is  a  finite  non-empty  set  determined  by  the  2ni  n  vertices  of  the 
rectangular  polyhedron  defined  by  0  s  <,  a.,  ¥-1,j.  Finally,  ,  de¬ 
fined  as  the  intersect  on  of  and  T.  ,  is  finite,  since  T1  Is  finite,  and 
non-empty,  since  at  least  x  *  0  belongs  to  the  intersection  ft,.  Since 
is  non-empty,  an  opt-ma  solution  to  profem  P  a’way^  exists. 

Bftandujig  opzi uuU-.a  G'ven  a  renam  node  i  of  the  solution  tree  with 

associated  sets  and  S^,  the  branching  is  denned  by  their  intersection 
with  the  sets 


h.r  '  (xij  /  \t  "  01 


h.~\  ■  U,J  '  x»  ■  V 


for  a  gi ven  i  =  s  and  j  =  t  .  The  sets  thus  defined  satisfy  the  first 
sufficiency  condition  of  Theorem  2.1,  that  is: 


V, 


PiV, 

r  c.r+l 


/  «st  ■  0,  « 


St 


=  a$}  = 


Also,  since  is  a  subset  of  (by  the  branching  operation)  then  from 
(5.7)  the  x  t  components  of  the  vector  elements  of  must  be  either  0 
or  a$.  Then  (V£>rPl  V^>r+] )  =  /  x$t  =  0  or  x$t  •  a$}  *  Qt 

and  the  second  condition  for  sufficiency  of  Theorem  2  i  is  also  satisfied. 
Finally,  since  at  each  branching  operation  one  variable  is  fixed  to 
each  one  of  its  poss’-ble  values,  the  fin-teness  of  the  number  of  variables 
assures  that  only  a  finite  number  of  branching  operations  are  required 
before  total  enumeration  of  the  elements  of  fi1  is  accomplished. 


Obviously,  no  capital  outlay  may  exceed  the  sum  of  all  the  budgets; 
if  so,  it  may  be  ruled  out  of  the  problem 
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5.5  THE  AUXILIARY  PROBLEM  AND  ITS  SUBALGORITHM 

At  each  iteration  of  the  branch  and  bcjnd  algorithm,  associated  with 
each  newly-generated  node  l  of  the  solution  tree,  a  continuous  muJUaJuj 
problem  A^  derived  from  P  must  be  solved. 

Denote  by  IQC  A  the  subset  of  arcs  (1,J)  e  A  of  the  network  G  for 
which  x^  *  0  (i,e  ,  Investment  on  project  1  rejected  at  period  j):  by 
IaC  A  the  subset  of  arcs  with  xfj  »  a,  (l,e.,  project  accepted  In  period 
j);  and  by  T,  the  set  of  "free"  arcs.  Then  the  auxiliary  problem  A^ 
takes  the  form 


Aj  :  Maximize 


Subject  to 


m  m 


‘VW 

L*  Ll 

1=1  j=l 

l'ij  'Ij 

m 

E  x. 

5BJ 

,  j*l.. 

1-1  1J 

2  xij 

s  a1 

,  1*1.. 

x^  =  0  ,  (i,j)  e  I 


* ,  j  >  0 


.  (I.j)  e  T 


o 


a 


This  is  a  transportation  type  linear  program  In  Inequality  form  with 
♦ 

some  prohibited  routes  and  where  maximization  is  sought-  Therefore,  thi 
problem  may  be  solved  by  any  available  transportation  algorithm.  In 
particular,  the  generalized  pnmal-duai  algorithm  of  Fulkerson  [4]  is 
perfectly  suited  for  trm  problem.  Indeed,  at  each  node  r  of  the  branch 
and  bound  tree,  the  solution  to  the  auxiliary  problem  of  the  unique 


The  arcs  (»,J)  e  A  for  which  x  *  a.  may  be  interpreted  as 

"  J 

prohibited  routes  if  is  first  subtracted  from  the  corresponding 
nodes  ?  and  j 
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predecessor  node  t  of  r  may  be  used  as  a  starting  flow..  The  out-of-kilter 
algorithm  wl1 1  then  reoptimize  this  flow  according  to  the  status  of  sets 
1Q  and  associated  with  node  r.  It  r  corresponds  to  the  branch  *  0, 
It  suffices  to  set  the  lower  and  upper  bounds  on  that  arc  equal  to  zero. 
If,  on  the  other  hand,  the  branch  corresponds  to  x^  -  aiS  the  lower  and 
upper  bounds  win  be  set  equal  to  a,,. 

Observe  aso  that  if  an  optimal  solution  to  Af  has  been  obtained, 
given  by 


* 

‘ij 


if  ( 1  *J )  t  I, 


-  < 


i  i » J 


I 


0  s  *,J  S  *, 


if  (1»J)  e  I 


then  a  feasible  solution  to  P,  provided  by  a  rounding  operation  on  x.  ,  is: 


K, 


-J 


j  •  ,f  x*j  =  °  °r  a( 

J  ,  ifOOyU, 


This  simple  operation  permits  the  ise  of  the  double  bounding  technique 
as  well  as  the  rejection  operation  of  tne  branch  and  bound  algorithm,  which 
we  proceed  to  enunciate: 


it 

STEP  1,  Set  i  =  1  and  create  node  1  Sol*e  A,.  If  x  is  such  that  all 
x  .  *  0  or  a  ,  stop;  the  solution  is  optimal.  If  at  least  one 

'  J 

*  * 

»  i  0  or  a  .  bound  node  1  with  U,  *  z  (1)  Round  node  one  to 
ij  i  1 

obtain  x_,  z(l).  Set  -  z ( 1 ) .  Here  Uj  >  L^.  Set  i  ■  i  +  1 
and  go  to  step  ; 


STEP  1  a)  BRANCH  Branch  from  bounded  node  t  Create  nodes  r  and  r  ♦  1 

and  directed  arcs  U.r)  and  U,r*i).  Select  any  x such  that 
*  i  J 

0  <  x^  v  for  node  t,  and  branch  with  x.^  *  0  and  . 

Solve  Ar  usmg  ihe  subalgorl thi.,  and  then  based  on  the 
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solution  to  Af.  If  Af  or  Ar+1  are  Infeasible,  exclude  them  from 
further  consideration, 

b)  ROUND.  Round  nodes  r  and  r  H  to  obtain  £  and  z_^, 

r  r+ 1 . 

c.l)  BOUND  FROM  EELOW.  Set  L.  =  max  [L.  , ,  z(r),  z(r+l )]- 
Reject  all  nodes  with  z  <  L^ . 

c.2)  BOUND  FROM  ABOVE.  Select  node  l  such  that  z*(E)  =  max 
*  k 

[z  (k)],  for  current  terminal  nodes.  Upper  bound  node  L  with 

U^  =  z  (El  If  L1  s  U|,  stop;  the  feasible  solution  that  provides 

the  lower  bound  is  optimal.  Otherwise  L.  <  U^.  Set  i  ■  1  +1  and 

go  to  step  i. 


5.6  ALTERNATIVE  FORMULATION  FOR  THE  MUtTI -KNAPSACK  PROBLEM 

In  this  section  we  present  an  alternative  formulation  for  the  multi¬ 
knapsack  problem  which  permits  the  solution  of  the  auxiliary  problems  of  the 
brancn  and  bound  tree  by  inspection. 

Associated  with  each  project  1  e  N,,  we  introduce  a  decision  variable 
y^  restricted  to  take  the  values  0  or  1,  which  indicate  rejection  or 
acceptance  of  the  project  i,  respectively. 

Then  problem  P  may  be  formulated  as  follows: 


?2  :  Maximize 


z  * 


m  n 

i  i  CjjX... 

i  =  l  j=l  J 

(5.9) 

m 

r  x,  .  <  B . 

t  j-*  1 ,  *  •  •  ,n 

(5.10) 

n 

1  xi  i  s 

jsl  ^  '  1 

.  1  =  1 ..  ,m 

(5.H) 

x  .  2  0  dr  a, 

'  J  > 

,  v-  1 ,  V 

(5.12) 

y  0  or  1 

,  v-  i 

(5.13) 

Subject  to 
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Here  constraints  (5,11)  and  (5,13)  establish  the  fact  that,  If 
project  i  is  accepted,  y^  *  1,  the  total  outlay  over  all  time  periods  Is 
a^.  In  addition,  constraints  (5.12)  guarantee  that  the  outlay  a^  will  be 
disbursed  ir,  only  one  of  the  stages  considered,. 

We  may  express  P2  *n  a  more  convenient  form  for  developing  a  solution 
method,  as  follows: 


Maximize 

z  -  2 
i*i 

2 

j--' 

°!j  'lj 

(5.14) 

m 

j=l,,,,.n 

(5.15) 

Subject  to 

r 

U 

1=1 

*i  j 

*Bj  * 

0  < 

Xi  j 

5  ai 

,  ¥  1,  .1 

(5.16) 

n 

(5.17) 

l 

j»1 

Xij 

/  «i  •  0 

or  1  ,  -V-  i 

Xij 

=  0 

or  a] 

.  -V-  i ,  j 

(5.18) 

We  observe  that,  if  the  discrete  constraints  (5  17)  ana  (5.18)  are 
relaxed,  the  resulting  linear  program,  (5  14)  subject  to  (5.15)  and  (5.16), 
may  be  solved  by  inspection  Indeed,  it  is  composed  of  n  mutually 
independent  linear  programs,  each  one  associated  with  one  time  period 
j  e  N2 

Under  the  assumption  of  non-negative  c,,,  and  after  ordering  the  c. . 

'j  *  J 

for  each  j  e  in  decreasing  order,  the  optimal  solutions  may  be  obtained 
by  an  expression  analogous  to  (3  25) 

The  branch  and  bound  algorithm  may  therefore  be  applied  directly  to 
problem  ?2  above 

At  each  step  of  the  algorithm,  one  variable,  not  currently  satisfying 
(5.18),  is  fixed  to  its  possible  values,  thus  defining  the  branching 
operation. 
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We  remark  that  for  a  certain  node  of  the  solution  tree  the  values  of 
y.j  calculated  according  to  (5  V7)  may  be  greater  than  one;  in  fact,  they 
may  be  as  large  as  n,  where  n  is  the  number  of  time  periods. 

Note  also  that  a  rounding  operation  may  be  performed  at  each 
iteration.  However,  if  the  solution  of  the  auxiliary  problem  results  in 
a  certain  project  1  with  capital  outlays  a1  in  various  time  periods,  more 
than  one  feasible  solution  to  Pg  be  obtained  by  application  of  the 
rounding  operation 

When  a  comparison  is  made  of  the  branch  and  bound  algorithms  developed 
in  Section  5  5  and  the  one  indicated  here,  we  may  point  out  the  following: 

The  first  formulation  P  requires  the  solution  of  a  network  flow 
problem  at  each  node,  as  opposed  to  the  solution  of  n  simple  linear  pro¬ 
grams  solved  by  inspection  when  the  formulation  Pg  is  used.  However,  the 
second  approach  in  general  requires  the  search  of  a  larger  number  of  nodes 
before  optimality  is  reached. 


5.7  OPTIMAL  ALLOCATION  OF  PROGRAMS  TO  PRIMARY  MEMORY 

We  conclude  this  chapter  with  the  formulation  of  a  problem  which  is 

related  to  the  multi -knapsack  case  and  which  arises  in  the  context  of 

allocating  programs  to  primary  memory  in  a  computer  system. 

Consider  a  set  of  m  items  of  size  a^,  i  *  1,...,  m  that  are  to  be 

loaded  into  n  knapsacks  of  capacity  B .,  j  *  1 ,. .. ,  n.  We  assume 
n  m  »> 

E  B.  >  E  a..  The  problem  is  to  assign  items  to  knapsacks  so  that  the 
J*1  J  '  1=1  1 

minimum  number  of  knapsacks  is  used.  The  use  of  each  knapsack  incurs  a 
fixed  cost  f.,  and  thus  the  total  cost  of  using  the  knapsacks  is  to  be 

J 

minimized 

The  problem  may  be  formulated  as  follows: 
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Minimize 

2  * 

n 

£ 

M 

fJ  yj 

(5.19) 

Subject  to 

m 

l 

i  *  i 

(5.20) 

n 

Z 

j  1 

;  *i 

(5.21) 

m 

f 

U. 

i  i 

til 

ai 

S  m  yt 

(5.22) 

*  0  or 

(5.23) 

X’j 

-  0 

or  a1 

(5.24) 

where  the  decision  variable  y^  is  associated  with  each  knapsack. 
Constraints  (5.22)  guarantee  that  no  f’ow  will  occur  from  nodes  i  e  N-j 
to  node  j  e  N2  i f  y^  =  0. 

Again,  problem  P3  may  be  solved  by  direct  application  of  the  branch 
and  bound  technique  We  shall  indicate  here  that  if  constraints  (5.23) 
and  (5.24)  are  relaxed,  the  resulting  linear  program  may  be  solved  by 
means  of  a  network  flow  algorithm. 

Indeed,  since  we  are  minimizing,  the  optimal  solution  to  P3  without 
discreteness  constraints  will  necessarily  satisfy  (5.22)  as  a  strict 
equal lty : 


yo 

?  ^ 

1-0  a! 


0* 


(5.25) 


Substituting  (5  25)  in  the  objective  function,  the  problem  to  bo  solved 
Is: 
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1  in  n  f 

Minimize  z  -  -  E  E  T1  x,, 

m  j-i  57  ,J 


m 

Subject  to  E  x, .  <  B 
1=1  1J  ' 


j"u  =  at  • 

which  obviously  corresponds  to  a  transportation  problem,  and  thus  a 
network  flow  algorithm  may  be  employed  for  its  solution. 
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CHAPTER  VI 

THE  MULTISTAGE  NETWORK  DESIGN  PROBLEM 

6.1  INTRODUCTION 

In  this  chapter  we  shall  propose  a  programming  model  aimed  at 
determining  an  optimum  transportation  network  development  plan  for  a 
metropolitan  area.  The  model  to  be  derived  synthesizes  the  best  network 
configuration  among  a  set  of  suggested  improvements  maintaining  ex¬ 
penditures  within  expected  future  budget  ceilings.  The  budgetary 
constraints,  projected  into  the  future  from  trend  studies  of  past  trans¬ 
portation  expenditures,  are  assumed  to  be  given  up  to  a  fixed  horizon  In 
a  predetermined  staging  sequence.  The  model  furthermore  assumes  the 
continuity  of  a  stable  technology  over  the  entire  period  of  Interest. 

The  figure  of  merit  selected  for  optimization  Is  the  total  user 
cost  over  all  time  periods.  The  required  Input  data  are  the  expected 
origin-destination  demands  for  all  time  periods  over  the  existing  network; 
the  subset  of  existing  links  selected  for  capacity  improvement  with  the 
corresponding  capital  requirement;  and/or  the  construction  costs  of 
specific  links  to  be  added  to  the  current  network  and  their  total  improve¬ 
ment  of  capacity  If  selected  for  construction.  Also  required  Is  the 
topology  of  the  existing  network,  with  link  capacities  and  estimated 
users  cost  per  link  for  all  periods  of  Interest. 

The  problem  described  Is  interpreted  as  a  capital  Investment  problem 
with  dependent  projects,  where  the  desirability  of  combinations  of  pro¬ 
jects  will  be  reflected  in  the  redistribution  of  flow  volumes  and  therefore 
in  a  reduction  of  total  users  cost.  The  projects'  interdependency  rela¬ 
tionship-  are  taken  Into  consideration  by  imbedding  into  the  basic  model 
a  network  flow  distribution  submodel.  In  this  fashion,  the  nrultistaged 
network  design  problem  is  interpreted  as  a  combined  network  flow  and 
capital  budgeting  nroblem. 
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The  various  topics  considered  In  this  chapter  are  organized  In  the 
following  manner:  we  begin  by  presenting  a  general  overview  of  the  urban 
transportation  planning  process  with  emphasis  on  the  various  classes  of 
network  improvement  evaluation  models,  their  characteristics  and  main 
drawbacks,  "ext,  a  discussion  of  the  various  levels  of  network  Improve¬ 
ments  and  a  .view  of  existing  models  for  each  type  of  Improvement  Is 
presented.  With  this  background  material,  the  basic  multlstaged  model  Is 
developed  in  terms  of  a  highly-structured  mixed-integer  linear  programming 
formulation.  The  structure  of  the  model  is  then  thoroughly  analyzed  in 
order  to  propose  a  convenient  optimization  technique  for  carring  out  its 
solution.  Tne  proposed  solution  procedure  is  the  partitioning  algorithm 
of  Benders  (cf.  Appendix  A)  which  fully  utilizes  the  decomposable  nature 
of  the  multi-stage  problem. 


6.2  THE  URBAN  TRANSPORTATION  PLANNING  PROCESS 

The  need  for  an  integrated  long-range  transportation  plan  for 
metropolitan  areas  has  been  widely  recognized  by  civil  engineers,  city 
planners,  economists,  sociologists,  city  officials,  etc.  in  the  postwar 
era,  as  a  result  of  the  explosive  increase  in  the  size  and  complexity  of 
urban  areas.  The  need  for  such  a  master  plan  has  been  officially  endorsed 
by  Congress  in  the  Federal -Aid  Highway  act  of  1962,  which  grants  federal 
aid  to  urban  areas  of  more  than  fifty  thousand  population,  provided  that 
their  projects  are  based  on  "a  continuing  comprehensive  transportation 
planning  process. . 

The  planning  process  is  primari1y  concerned  with  forecasting  future 
demand  for  transportation  in  a  certain  study  area,  as  well  as  planning 
transport  facilities  that  provide  a  satisfactory  level  of  service  while 
maintaining  the  co '•res ponding  capital  expenditures  within  expected  future 
budget  ceilings. 

Comprehensive  studies  such  as  the  Chicago  Area  Transportation  Study 
(CATS),  Penn-Jersey  Transportation  Study,  etc.,  have  been  carried  out 
from  a  systems  viewpoint;  the  attempt  to  consider  all  the  interacting 
elements  that  affect  the  demand  for  transportation,  and  to  plan  new 
f?:. titles  in  the  light  of  their  interaction  with  the  existing  network 
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(as  opposed  to  making  local  decisions  or  accepting  small-scale  palliative 
solutions)  for  a  genera!  description  of  different  Issues  akin  to  urban 
transportation  studies  see  Moyer  [1], 

The  relation  of  urban  development  to  demand  for  transportation  and 
the  effect  of  new  facilities  upon  demand  patterns  have  been  carefully 
identified.  Past  research  has  focussed  on  the  characteristic  steps  of 
the  planning  process;  quantitative  approaches,  such  as  use  of  mathematical 
models  and  techniques,  have  been  suggested  for  each  step  of  the  planning 
process,  and  much  experience  and  momentum  has  been  gained  from  these  studies. 

Most  transportation  planning  models  are  based  on  expediting  the 
extrapolated  trend  of  economical  and  environmental  development 
Although  the  transportation  studies  carried  on  for  various 
metropolitan  areas  had  to  treat  different  problems  according  to  the  spe- 
cirir  areas  of  interest,  they  present  virtually  the  same  pattern  in  their 
solution  jpproach.  This  pattern  indicates  the  fundamental  steps  of  the 
transportation  planning  process  which  we  proceed  to  enumerate, 
i)  Inventories  for  base  year: 

These  consist  of  inventory,  for  a  reference  year  Tfc  (base  year),  of 
the  relevant  factors  that  will  affect  the  future  demand  for  transportation. 
The  inventories  usually  considered,  mostly  based  on  censuses,  are  listed 
below. 

i  land  use  inventory 
ii  population  inventory 
n i  transportation  inventory 
iv  trend  of  transportation  expenditures 

ii)  Inventories  for  target  year:  These  a  e  developed  by  fore¬ 
casting,  for  the  taxgzt  yiai  T^  (usually  a  20  or  25  year  interval).,  the 
changes  in  the  bait  t/eaa  inventories. 

This  forecasting  is  usually  attained,  for  each  type  of  inventory,  by 
means  of  prediction  models  of  varying  sophistication  Martin,  Meirmott 
and  Bone  [2]  present  an  analysis  and  detailed  description  of  various 
models  often  used  in  the  planning  process. 

It  is  interesting  to  observe  that  these  first  two  steps  are 
invariably  required  for  an  integrated  study  of  the  development  and 
improvement  of  any  kind  of  facilities  in  a  metropolitan  area. 
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iii)  Transportation  Analysis: 

Based  on  the  demand  for  transportation  at  Tg  and  on  both  the  base  year 
and  target  year  inventories,  the  future  demand  for  transportation  is  fore' 
cast,  and  a  master  transportation  plan  is  developed.  This  analysis  con¬ 
stitutes  the  core  of  the  transportation  planning  process  and  its  accuracy 
will  be  a  direct  function  of  the  accuracy  and  completeness  of  the 
forecasting  analysis. 


6.3  TRANSPORTATION  ANALYSIS 

We  shall  briefly  mention  the  now  well-established  steps  into  which 
the  transportation  analysis  phase  is  subdivided.  For  each  step,  well- 
developed  models  are  available,  and  a  substantial  amount  of  research  is 
currently  underway  seeking  to  verify  and  improve  the  accuracy  of  such 
models. 

i)  T-'u.p  Gtn.VLaXJ.on.  The  purpose  of  trip  generation  is  to 
determine  the  number  of  trips  starting  (or  ending)  in  a  particular  zone 
of  the  study  area  for  specified  future  years. 

ii)  T>u.p  V^tnXbuXXon.  Tr'p  distribution  is  the  process  of 
assigning  destinations,  by  means  of  a  distribution  model  such  as  the 
gravity  model,  to  the  trips  generated  in  each  zone  of  the  study  area. 

iii)  Modal  Split.  The  modal  split  analysis  is  used  to  estimate  the 
future  breakdown  of  trips  among  the  available  transportation  modes.  The 
models  most  frequently  employ  multiple  regression  analysis,  and  are  used 
to  predict  future  modal  split  for  the  modified  values  of  the  input 
variables;  this  clearly  implies  that  no  major  changes  in  transportation 
technology  are  expected  during  the  period  of  interest. 

The  modal  split  phase  assigns  each  future  traffic  demand  by  mode  to 
the  corresponding  transportation  network  for  that  mode. 

iv)  Tia^-tc  Ai<su}f»nen*.  The  objective  of  the  traffic  assignment 
is  to  determine  flow  patterns  in  specific  transportation  networks,  where 
flows  are  associated  with  the  different  modes  adopted  in  the  planning 
process.  This  step,  being  of  special  interest  for  the  present  work,  will 
be  treated  in  more  detail  in  forthcoming  sections. 
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v)  T/iampoitation  System  Evaluation,  The  traffic  assignment  step 
is  usually  performed  for  each  mode  and  for  alternative  transportation 
networks,  with  the  main  objective  of  obtaining  substantial  information 
on  the  relative  performance  of  the  alternatives.  This  will  hopefully 
permit  the  rational  selection  of  the  transportation  system  that  will  best 
meet  the  future  demand  with  a  suitable  level  of  service. 

The  evaluation  of  the  various  alternatives  is  usually  done  by 
standard  techniques,  such  as  cost-benefit  analysis  or  rate-of-return 
method.  In  a  forthcoming  section  this  evaluation  step  will  be  analyzed 
and  identified  as  a  capital  budgeting  problem  which  can  be  systematically 
and  quant' tifveiy  atta:ked 

The  output  of  the  evaluation  phase,  possibly  obtained  after  several 
iterative  cycles  of  the  total  process,  will  be  the  desired  long-range 
urban  transportation  plan  for  the  area  of  interest 


6.4  THE  TRAFFIC  ASSIGNMENT  PROBLEM 

In  the  context  of  transportation  planning,  the  term  traffic 
assignment  means  the  determination  of  flow  volumes  on  the  links  of  a  given 
transportation  network,  where  volumes  per  unit  of  time  are  specified  be¬ 
tween  each  zonal  pair  in  a  set  of  origin-destination  pairs.  The  traffic 
assignment  permits  the  evaluation  of  the  performance  of  network  alternatives. 

The  question  of  how  the  flow  distributes  itself  over  the  network 
constitutes  one  of  the  most  important  issues  in  transportation  planning. 

Two  different  criteria,  enunciated  by  Wardrop  [3]  and  formalized  in 
mathematical  form  by  Beckman  et  al  [4]  and  Charnes  and  Cooper  [5],  nave 
initiated  the  development  of  two  major  classes  of  traffic  assignment 
models.  These  are  generally  given  the  titles  of  dcsc.'uptue  (predictive) 
and  nc”unat<ve  (prescriptive)  models  Each  Wardrop  postulate  suggests  that 
the  flow  distributes  itself  over  the  netwo.K  according  to  one  of  two 
contrasting  extremal  principles: 

0  Postulate  of  equal  travel  times:  for  a  flow  assignment,  the 
travel  time  between  any  two  points  on  the  network  will  be  the  same  on  all 
routes  used  and  less  than  the  travel  time  on  any  other  path  joining  the 
same  two  points 
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11)  Postulate  of  overall  minimization:  for  an  optimal  flow  assignment, 
the  average  travel  time  for  all  users  of  the  network  attains  Its  minimum 
value. 

VucAiptive.  Tu^ic  ModeZi 

This  family  of  traffic  assignment  models  Is  based  on  Wardrop's 
principle  of  equal  travel  times  The  computer  implementation  of  such 
models  has  acquired  great  momentum  as  a  result  of  their  use  In  transpor¬ 
tation  studies  of  major  metropolitan  areas  during  the  early  sixties. 

These  programs  implicitly  use  the  game  theory  model  of  Charnes  and  Cooper, 
where  all  travelers  seek  to  minimize  their  own  travel  time. 

The  flow  distribution  is  achieved  by  Iteratively  assigning  traffic 
from  each  origin  node  to  all  destinations  according  to  current  shortest 
path-routes.  After  completion  of  each  Iteration,  the  resultant  travel 
times  on  links  are  updated  according  to  their  current  loads  and  the  origins 
will  again  take  turns  assigning  portions  of  their  flows. 

The  descriptive  models  used  In  different  transportation  studies 
present  variations  In  their  actual  calculation,  but  they  are  all  based  on 
the  principles  indicated  above-  In  [6]  and  [7]  the  reader  will  find  a 
complete  description  and  comparison  of  the  various  models  in  use  today. 

Normative  Tia^c  Aaa-tgnm ent  Model* 

This  class  of  models  is  based  on  Wardrop’s  postulate  of  overall 
minimization  and  on  the  traffic  flow  analysis  of  Beckman  et  al.  and  Charnes 
and  Cooper:  flows  distribute  themselves  so  as  to  minimize  the  total 
travel  time  in  the  system,  as  opposed  to  Individual  travel  times. 

This  optimization  problem  has  been  formulated  by  Charnes  and  Cooper 
[5],  for  congested  networks,  as  a  non-linear  programming  problem;  the  non¬ 
linearity  results  from  the  fact  that  ’ink  travel  times  increase  non- 
llnearly  w’th  flow  volumes  They  further  simplify  their  model  by  suggesting 
a  plece-wijo  1 mearfzation  of  cne  travel  time-volume  relationship, 
accomplished  by  introducing  multiple  capacitated  arcs  with  Increasing 
travel  times.  The  resulting  model  is  a  linear  program  known  as  the 
multlcopy-cost-mir.irnization  network  flow  problem.  This  problem  has  been 
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thoroughly  analyzed  and  exploited  by  Pinnell  and  Satterly  [8]  and  by 
Her^dorfer  [9]. 

Jorgensen  [10]  has  studied  both  classes  of  traffic  assignment  models, 
and  shows  that  for  the  uncongested  case  (rural  networks),  both  the 
descriptive  and  normative  solutions  give  the  same  flow  distribution 
pattern 

The  actual  computer  implementation  of  normative  models  requires  a 
linear  programming  routine  capable  of  handling  a  potentially  large  number 
of  constraints;  or  alternatively,  with  the  additional  capability  of  ex¬ 
ploiting  the  highly-structured  form  of  the  model  by  conveniently  decompos¬ 
ing  the  proolem  into  mo'c  tractable  subprograms. 


6.5  TRANSPORTATION  NETWORK  IMPROVEMENTS 

The  main  goal  of  the  traffic  assignment  is  to  determine  the  level  of 
service  provided  by  a  given  network  for  a  set  of  demands  previously  spe¬ 
cified.  When  the  demand  expected  for  the  target  year  is  assigned  to  the 
network  configuration  of  the  basic  year,  it  is  likely  that  the  latter  will 
not  provide  a  satisfactory  level  of  performance.  This  condition  will  be 
reflected  in  the  final  assignment  by  an  excessive  number  of  links  operating 
under  congestion. 

On  the  other  hand,  if  new  urban  areas  are  expected  to  be  developed 
by  the  target  year,  the  network  will  have  to  be  expanded  to  provide 
transportation  facilities  to  these  areas. 

This  situation  cleanly  calls  for  a  network  improvement  plan  to  meet 
the  forecasted  demand,  making  use  of  the  limited  capital  resources  expected 
to  be  available  for  such  purposes  during  the  period  of  interest.  Various 
levels  of  improvement,  some  of  which  are  listed  below,  may  be  undertaken 
to  cope  with  the  increasing  demand  pressures. 

i)  Augmentation  of  capacity  in  existing  links.  This  improvement 
may  be  realizable  by  various  means,  ranging  from  enforced  parking  re¬ 
strictions  1 n  certain  arteries  to  new  lane  construction  and  more  expedient 
traffic  control  systems 
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It)  Rearrangement  of  one-way  and  two-w\  streets  to  provide  an 
optimal  configuration. 

Ill)  Addition  of  new  links  to  the  existing  network. 

1 v)  For  public  transportation,  construction  of  new  terminal 
facilities  and  links  to  connect  these  facilities  with  already  existing  ones. 

In  practice,  the  filial  long-range  transportation  plan  may  call  for 
a  mixed  strategy  utilising  various  modes  of  network  improvement.  It  is 
obvious  that  a  transportation  planner  has  to  analyze  a  large  number  of  im¬ 
provement  alcernat'ves,  before  a  final  plan  is  adopted.  The  two  classes 
cf  traffic  assignment  models  studied  previously  provide  totally  different 
approaches  to  solving  such  synthesis  problems. 


6.6  NETWORK  SYNTHESIS  VIA  DESCRIPTIVE  MODELS 

To  describe  the  synthesis  solution  when  descriptive  models  are 
employed,  let  us  assume  that  a  specific  set  of  links  proposed  for  con¬ 
struction  constitute  tue  type  of  improvement  prescribed. 

It  is  obvious  that  each  project  may  not  be  analyzed  independently  of 
the  others,  since  the  total  network  performance  is  highly  dependent  on  the 
combination  of  projects  considered,  On  the  ether  hand,  if  m  is  the  number  of 
possible  link  additions,  ?.  different  alternatives  exist,  and  its  exhaus¬ 
tive  analysis  is  clearly  impossible  for  even  moderately  large  m.  The 
usual  practice  in  the  economic  evaluation  of  traffic  networks  is  to  select 
a  pUoxi,  a  small  subset  of  the  potentially  large  number  of  alternative 
networks  and  accept  the  one  that  provides  the  best  "measure  of  effective¬ 
ness". 

To  determine  that  measure  of  effectiveness,  a  traffic  assignment  is 
required  for  each  alternative  network  as  provided  by  a  given  descriptive 
model.  The  output  of  the  traffic  assignment  (average  daily  traffic  for 
each  link)  may  be  converted  into  users’  cost.  The  accumulated  users' 
cost  for  the  entire  network,  and  the  total  capital  investment  for  the 
plan  presently  considered,  are  the  parameters  needed  for  estimating  a 
measure  of  effectiveness  for  that  project.  A  detailed  description  of  the 
various  elements  required  in  such  a  process,  as  well  as  procedures  and 
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methods  to  obtain  them,  may  be  found  in  the  work  of  Halkalis  and  Joseph 
[11].  When  the  process  just  described  has  been  completed  for  the  subset 
of  plans  under  analysis,  the  usual  practi  e  is  to  apply  a  benefit-cost 
ratio  analysis,  a  rate  of  return  on  marginal  investment  analysis,  or  some 
other  classic ia!  economic  method,  to  determine  the  best  alternative  among 
those  which  have  been  preselected  for  consideration. 

When  a  budgetary  constraint  is  imposed  on  improvement  expenditures, 
the  synthesis  problem  may  be  solveo  by  direct  application  of  optimization 
methods  for  combinatorial  problems.  In  particular,  a  direct  search  tech¬ 
nique  or  an  implicit  enumeration  method  may  be  in  order.  We  conjecture 
here  that  it  rriuy  be  des'rabie  to  apply  an  implicit  enumeration  technique 
as  described  below  First,  we  assume  th. v  as  the  number  of  links  added 
to  the  network  increases,  the  total  user  ;.nst  declines-  Let  the  MOE  be 
the  user  cost,  with  6  the  budgetary  ceiling  v  capital  investment,  and 
an  m  component  binary  vector  associated  with  the  acceptance  or  rejection 
of  the  links  considered  for  construction.  Hence,  the  new  link  addition 
problem  may  be  expressed  in  terms  of  B  and  the  project  costs  as 

S  :  Minimize  z  =  f(y)  (6.1 ) 

m 

ojbject  to  Eg.y.  <  B  (6,?) 

j=l3  J  ' 

y .  =  0  or  1  (6.3) 

j 

where  z  is  the  total  users  cost  as  a  function  of  the  vector 

Problem  S  is  a  constrained  optimization  problem  that  may  be 
interpreted  as  capital  rationing  for  dependent  projects.  The  dependency 
appears  in  the  objective  function  (61/,  which  cannot  be  expressed  in 
closed  mathematical  form,  but  can  only  be  evaluated  as  a  result  of  a 
traffic  assignment  for  each  vector  £,  (each  network  configuration) 
considered. 

SOLUTION  METHOD.  We  shall  propose  an  implicit  enumeration  technique 
based  on  the  general  algorithm  of  Chapter  II,(See[18]  for  a  complete 
presentation),  where 
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VI 

o 

•n 

& 

(6.4) 

m 

^ j  B-  yi  * 0  or  11 

(6.5) 

s,nr, -t, 

(6.6) 

The  auxiliary  problem  becomes: 

Aj  :  Minimize  z  -  f(^) 

Subject  to  0  <  y^  <  I  ,  j  e  3 

yj  *  0  .  i  t  J0 

yi  *  1  •  J  E  Ji 


where  3  is  the  set  of  free  links,  JQ  the  set  of  rejected  links  and  the 

set  of  accepted  links.  Under  the  assumption  that  the  value  of  z  does  not 
increase,  as  the  number  of  links  added  to  the  network  increases,  the 
optimal  solution  to  the  optimization  problem  A^  is 


y. 


0,  if  j  e  JQ 
1 ,  otherwise 


(6.7) 


where  the  value  z  (l)  is  determined  after  the  output  of  a  computer  program, 
which  performs  the  descriptive  traffic  assignment  and  converts  the  link 
traff’C  volumes  into  user  cost,  is  obtained.  The  branch  and  bound  al¬ 
gorithm  may  then  be  stated  as  follows: 
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STEP  1.  Set  i  -  1,  Generate  node  1  by  solving  a  traffic  assignment  with 

t  * 

Vj  -  1.  ¥  j,  Let  z  (!)  be  the  total  user  cost.  Calculate 
*  m  *  * 

B  -  la^j  If  B  <  B,  stop;  the  network  configuration  Is 

optimal.  Otherwise,  bound  node  1  with  L^  =  z*(1).  Set  1*1+1 
and  go  to  step  1, 

STEP  i 

a)  BRANCH  Branch  from  bounded  nooe  l.  Select  one  yk  to  be  fixed 
to  zero  and  one  Create  nodes  r  and  r  +  1  and  directed  arcs  U,r)  and 

(£,  r  t  l)  So've  the  traffic  assignment  corresponding  to  Ap  with  y^  *  0, 
adding  k  to  the  set  JQ  Solve  the  traffic  assignment  corresponding  to 
A^,  with  yk  -  1 ,  adoing  k  to  Jj, 

b)  BOUND,  Select  node  l  such  that  z *(t)  -  min  (z*(r)},  for  current 

m  *  r 

terminal  nodes.  If  I&y,  <  B  for  node  lt  stop;  the  solution  associated 
j=lJ  J  ’ 

with  node  l  is  optimal  Otherwise,  set  1  *  1  +  1  and  go  to  step  1. 


6.7  NETWORK  SYNTHESIS  VIA  NORMATIVE  MODELS 

The  important  advantage  of  normative  models  lies  In  theiv  flexible 
handling  of  synthesis  problems,  since  the  intrinsic  nature  of  optimization 
problems  is  such  that  a  convenient  solution  technique  takes  care  of  the 
combinatorial  aspects,  and  finally  selects  the  best  project  combination. 

A  substantial  amount  of  research  has  been  undertaken  in  this  area, 
and  various  model  formulations  have  evolved  from  the  study  of  various 
types  of  network  improvement  problems. 

The  technique  of  continuous  augmentation  of  capacity  on  existing  links 
has  been  formulated  by  Garrison  and  Marble  [12]  and  by  Quandt  [13].  In  v 
the  latter  model,  the  construction  ..ost  appears  as  a  budgetary  constraint, 
rather  than  as  part  of  the  objective  function,  as  treated  by  Garrison  and 
Marble. 

Hershdotfer  [9]  studied  the  optimal  one-way  and  two-way  street 
configuration  by  extending  Chcrnes  and  Cooper’s  multi-copy  network  model 
by  an  ad  kvz  introduction  oi  decision  variables  Into  the  model. 
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Hershdorfer,  and  also  Roberts  and  Funk  [14]  have  used  Dantzlg's 
scheme  of  Introducing  decision  variables  In  the  upper-bounding  constraints 
on  certain  links,  thereby  obtaining  a  suitable  formulation  for  the  new 
link  addition  problem,  Roberts  and  Funk  consider  rural  network  Improve¬ 
ment  subject  to  a  budgetary  constraint  as  opposed  to  Hershdorfer,  who 
essentially  assumes  an  Infinite  budget  and  congested  networks.  Recently, 
Ridley  [15]  has  developed  a  combinatorial  approach  which  he  calls  the 
"method  of  bounded  subsets"  for  solution  of  the  discrete  augmentation  of 
capacity  problem 

The  branch  and  bound  algorithm  developed  In  Section  6.6  Is  equally 
applicable  to  the  new  Unk  addition  synthesis  problem  when  normative 
models  are  employed.  In  this  case,  the  subprograms  correspond  to  multi¬ 
copy  network  flow  problems,  which  can  be  sclved  by  means  of  a  decomposition 
form  linear  programming  code. 

The  simultaneous  optimal  node  and  link  selection  for  an  urban  public 
transportation  network,  subject  to  a  budgetary  constraint,  has  been  solved 
by  Ichblah  [16]  by  means  of  a  parametric  branch  and  bound  technique.  His 
model  does  not  directly  consider  flow  volumes  on  the  proposed  network. 

The  set  of  models  described  above  study  network  improvement  problems 
for  a  single  time  period  (base  year  to  target  year).  In  fact,  the  budget 
available  for  transportation  investments  is  commonly  appropriated  in  a 
multi-stage  manner.  Although  the  models  Indicated  may  be  applied  suc¬ 
cessively  for  various  time  increments,  what  the  long-range  transportation 
plan  calls  for  is  a  sequence  of  Improvements  of  the  traffic  network  so 
that  a  convenient  figure  of  merit  is  optimized  over  the  total  sequence  of 
planning  periods.  The  purpose  of  this  chapter  Is  to  formulate  a  normative 
model  that  represents  the  goals  indicated  above,  for  different  types  of 
Improvements. 
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6.8  NETWORK  IMPROVEMENTS  OVER  TIME 

Based  on  the  discussion  of  previous  sections,  we  may  conclude  that 
the  preparation  of  a  long-range  transportation  plan  for  a  target  year  has 
been  sufficiently  studied,  and  that  a  variety  of  mathematical  models  are 
available  to  conveniently  attack  the  problem.  Considering  the  fact  that 
a  master  plan  would  be  actually  implemented  in  a  stage  by  stage  fashion, 
and  that  available  funds  for  transport  expenditures  are  usually  appro¬ 
priated  in  fixed  amounts  for  each  planning  period,  a  model  taking  this 
staging  into  account  seems  more  appropriate. 

Research  on  network  improvement  overtime  has  been  done  by  Kalaba 
[16]  for  communication  networks.  The  problem  that  he  considers,  howeve-  . 
is  continuous  augmentation  of  existing  links'  capacities.  Roberts  [17] 
has  studied  the  multistaged  link  addition  problem  and  proposes  a  solution 
method  based  on  solving  each  stage,  commencing  from  the  last  one,  with  a 
budget  equal  to  the  sum  of  the  budgets  up  to  the  stage  being  considered. 
Links  not  accepted  in  the  last  period,  are  deleted  from  further  considera¬ 
tion.  His  solution  does  not  necessarily  provide  an  optimum  when  the  goal 
is  to  minimize  a  figure  of  merit  over  the  entire  horizon. 

Before  developing  a  noimative  model  for  the  mult1' stage  link  addition 
problem,  we  shall  mention  certain  important  aspects  of  the  problem. 

In  the  preparation  of  a  transportation  plan,  before  decisions  can  be 
made  regarding  facility  improvements  which  are  feasible  in  terms  of  cash 
flow,  a  preliminary  planning  of  new  facilities  is  required.  The  study  of 
deficiencies  in  capacity  provides  a  basis  for  such  preliminary  design. 

We  shall  assume  that  a  set  of  possible  new  facilities,  from  which  no 
optimum  plan  or  subset  is  to  be  selected  has  been  establi c^d. 

We  assume  further  that  the  construction  costs  for  a  specific  type 
of  facility  have  been  previously  obtained.  This  is  obviously  difficult 
since  in  order  to  obtain  them,  the  facility  must  be  located;  and  to  es¬ 
timate  cost,  certain  standards  must  be  fixed,  which  depend  in  general  on 
the  flow  volumes  likely  to  use  the  facility. 

Finally,  the  future  demands  for  transportation  required  by  the  model 
have  been  derived  from  forecasted  land  use  patterns,  but  the  new  facili¬ 
ties  provided  in  the  planning  period  will  in  turn  modify  the  land  use 
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development  pattern,  We  shall  not  consider  this  Interaction  directly,  but 
It  could  be  treated  with  an  Iterative  application  of  the  model. 

With  these  assumptions  made  we  proceed  to  develop  an  optimization 
model  that  permits  us  to  determine,  for  normative  traffic  behavior,  the 
best  Improvement  plan  In  time  and  space. 


6.9  ANALYTICAL  FORMULATION 

Consider  the  graph  G  =  [R,  A]  consisting  of  nodes  which  are  denoted 
in  any  order  by  the  sequence  of  numbers  1,  2,,..,N  and  of  directed  arcs 
(i.  j)  joining  nodes  of  the  set  R  The  set  of  all  arcs  is  denoted  by  A, 
and  will  be  partitioned  into  two  sets  X  and  Y  such  that  XUY  *  A.  The 
set  X  contains  the  set  of  all  arcs  of  the  existing  network  of  interest. 

The  set  Y  contains  all  arcs  which  form  the  set  of  proposals  to  be  added 
to  the  network  over  n  time  periods.  Note  that  if  X  =  *  we  are  confronted 
with  a  complete  synthesis  problem. 

Let  the  amount  of  flow  of  copy  a  (here  a  copy  associates  all  of  the 
traffic  flowing  from  or  to  a  specific  origin  or  destination)  associated 
with  arc  (i,  j)  e  A  at  stage  k  be  x?^.  Denote  by  the  discounted 
unit  cost  of  travel  on  arc  (i,  j)  at  stage  k,  and  by  u^  the  capacity 
or  upper-bound  on  the  flow  over  arc  (i,  j)  at  stage  k. 

For  each  (i,  j)  c  Y,  let  a^k  be  the  capital  outlay  required  to 

build  arc  (i,  j)  if  selected  for  construction  at  period  k. 

Denote  by  >  0  the  net  amount  of  flow  into  node  i  of  copy  a  at 

time  period  k,  (or  r^k  <  0  if  the  net  flow  is  out),  and  by  E,  the  node¬ 

arc  incidence  matrix  which  describes  the  network  G.  The  total  budget 
ceilings  available  at  time  period  k  will  be  denoted  by  Bk.  Let  n  be  the 
number  of  time  periods  and  N  the  total  number  of  copies. 

The  problem  of  optimally  selecting  link  proposals  for  construction 
is  that  of  satisfying  the  budgets -y  constraints  at  each  period  and  mini¬ 
mizing  the  total  user  cost  over  the  entire  interval.  It  may  be  formulated 
as  follows: 
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N  n 

M  :  Minimize  z  ■  Z  l  Z  c<41, 

a«1  k=1  (l.j)eA  1JK 

xa 

x1jk 

(6.8) 

Subject  to  ^  e,j  x“Jk  •  r“k 

ifl.a,  k 

(6.9)  ‘ 

J,  xtjk  *  u1jk 

.  (1  »j)eX,  ¥k 

(6.10) 

x“jk  s  “ijt  **jk 

Ot  j 

,  (i.jM,  V-  k 

(6.11) 

yi jk  "  yijk+1  -  0 

,  (i.j)eY,  k=l , 

. . .  ,n-l ,  n  >  1 

(6.12) 

(*) 

(iJ)eY  aiJk  <y^k  ' 

W£Bk-*k  <6-13> 

4jk  -  0 

(6.14) 

*1jk  '  0  °r  1 

(6.15) 

In  this  formulation,  constraints  (S-9)  represent  the  conservation  of 
flow  equations  for  all  copies  and  all  time  periods,  with  e,.  being  the 

'  J 

corresponding  element  of  the  node  arc  incidence  matrix  E. 

The  constraints  (6  10)  constitute  the  upperbounding  constraints  on 
the  sum  of  all  copy-flows  utilizing  originally  existent  arcs  (i,j)  e  X. 

For  proposed  arcs,  (i,j)  c  V,  a  set  of  decision  variables  y^  has  been 
introduced  which  can  take  the  binary  values  1  or  0  as  indicated  by  (6.15), 
depending  on  whether  or  not  arc  (i,j)  e  Y  is  available  for  use  at  time 


For  n*l,  constraints  (b . 12 )  have  no  particular  meaning  and  may  be 
dropped  from  further  consideration. 
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period  k.  Constraints  (6.11)  therefore  guarantee  that  for  a  non-cons tructed 
arc,  the  corresponding  flows  will  vanish  (y^  =0),  or  otherwise  that 
they  do  not  exceed  the  provided  capacity  (y^  -  1) 

The  set  (6.12)  acts  as  a  "turn-on  switch",  guaranteeing  that  if  a 
certain  link  Is  adopted  for  investment  at  time  period  s  (l.e. ,  y^s  *  1, 
with  y^  =  0,  k  <  s),  It  will  remain  available  for  utilization  in  sub¬ 
sequent  periods  (i.e.,  y^k  *  1,  k  >  s). 

Constraints  (6.13)  represent  the  budgetary  constraints;  here 
y^Q  =  0  for  all  (i,j)  e  Y.  The  difference  of  the  decision  variables  for 
two  subsequent  time  periods.  In  addition  to  (6.12),  guarantee  that  a  single 
capital  outlay  is  disbursed  for  each  project. 

If  the  problem  being  considered  calls  for  multiple  outlays  once  a 
link  has  been  selected  for  construction,  e.g,  when  maintenance  costs  are 
considered,  it  suffices  to  modify  the  budgetary  constraints  (6.13)  and 
replace  it  by 


I 

(1,j)eY 


*1jk  yijk  *  8k 


¥  k 


(6.16) 


Finally,  relation  (6.14)  simply  expresses  the  non-negativity 
conditions  on  the  arc  flows  for  all  time  periods. 

REMARKS.  We  observe  that  for  the  single  time  period  case,  the  index  k  may 
be  dropped;  constraints  (6.12)  will  no  longer  have  any  meaning  and  may 
also  be  dropped.  Constraints  (6.13)  reduce  to  a  single  budgetary  constraint 
and  problem  M  becomes  the  link-addition  problem  as  formulated  by  Roberts 
[17],  except  for  the  fact  that  construction  costs  are  not  part  of  the 
objective. 

If  both  indices  a  and  k  are  relaxed,  the  resulting  model  becomes  a 
single  period  problem  of  capital  investment  in  links  of  a  general  homo¬ 
geneous  commodity  network. 

This  problem  is  somewhat  similar  to  the  knapsack  problem  considered 
in  Chapter  IV.  The  main  difference,  however,  is  that  the  payoff  function 
for  a  certain  combination  of  links  may  not  be  determined  until  a  cost 
minimization  network  flow  problem  for  the  configuration  under  analysis  is 
solved. 
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Finally  we  remark  that  the  objective  function  is  linear  and  represents 
the  total  user  cost  over  the  enti  ^  interval  of  interest.  Therefore,  model 
M  in  its  most  general  toon  is  a  (0-1)  mixed-integer  linear  program  of  a 
very  complex  nature,  as  may  be  immediately  recognized,  but  with  important 
structural  characteristics  that  we  shall  idenr fy  in  the  following 
section. 


6-10  STRUCTURAL  CHARACTERISTICS  OF  THE  MODEL 

Let  us  assume  that  the  node-arc  incidence  matrix  E  describing  the 
network  G  =  [Fi,  A]  is  constructed  in  such  a  way  that  all  arcs  (i.j)  e  X 
occupy  the  first  part  of  the  matrix,  while  arcs  (i.j)  e  Y  will  be 
associated  with  the  remaining  columns  of  E.  Accordingly,  we  define  the 
following  partition  for  E,  E  *  [E,  E]. 

Let  x£  and  be  the  flow  vectors  for  arcs  (i.j)  e  X  and  v i ,j)  e  Y 
respectively,  for  copy  a  m  time  period  k.  Denote  by  the  vector  of  all 
decision  variables  at  time  period  k,  by  r£  the  demand  vector  for  copy  a  In 
period  k,  and  by  the  upper  bound  vector  on  the  flow  of  arcs  (i.j)  e  X 
at  time  period  k.  Finally,  according  to  the  partition  defined  for  E,  let 
<£  and  c£  be  the  user  cost  vectors  for  arcs  (i.j)  e  X  and  (I.j)  e  Y 
respectively,  for  copy  a  in  time  period  k. 

Our  model  M  may  then  be  rewritten  in  the  condensed  form  depicted  in 
Table  6-1.  Here  I  is  the  identity  matrix,  is  a  diagonal  matrix  having 
the  upperbounds  u  •.  for  (i.j)  r.  Y  as  diagonal  elements,  and  a.  is  the 

1  jK  -“tv 

vector  representing  capital  outlays  in  period  k  for  all  (i.j)  e  Y.  The 
non-negat'vity  conditions  on  the  flows  and  che  binary  values  of  the  y^, 
although  not  expl-citly  indicated  in  Table  6-1,  are  to  be  satisfied. 

The  arrangement  of  the  variables  in  Table  6-1  is  highly  suggestive 
of  a  partition  ’nto  two  sets,  the  first  embodying  the  derision  variables 
¥  k,  and  the  second  a'-’  the  How  variables  for  all  time  periods. 
Furthermore,  Table  6-1  presents  a  similar  structure  to  that  of  the  class 
of  problems  presented  in  Appendix  A,  (see  Page  A-3),  with  additional 
simplifications  Indeed,  using  the  notation  of  the  Appendix,  we  observe 
that  all  me  B  mat- ices  are  identical  in  our  problem  and  are  highly 
structured  as  we1’,  suggesting  that,  additiona1  e  »p'  o  ’  tation  possible. 


106  THE  MULTISTAGE  NETWORK  DESIGN  PROBLEM 


*1  *2 


-U„ 


Al 

'-a1  *2 


I  -I 


JmIuMM-  jg[ 


e  e 


a  A 

E  E 


e  e 


i  i 
i  i 


“-w 


i  -i 


t]C]  CZr2 
-1£1  -)-] 


TTi 

L  C 

E  E 

E  g 

II  I 

. — l . I _ L 

'$i 


-1*1  -2*2 
xV  xMxM 
-n-n  -n-n 


eA 

E 


E  E 


i  I 
I  I 


Jlcl  A* 

-n-fl  -n-n 


-rN$N 

*n®n 


E  E 


at 

tt- 

£i 

at 

A 

a 

-i 

* 

s 

X1 

a 

4 

a 

4 

a 

rN 

< 

HZ 

s 

m 

J 

-ft 

a 

0 

a 

4 

< 

-u* 

< 

C 

$ 

B1 

< 

B2 

• 

< 

• 

Bn 

< 

0 

• 

! 

• 

0 

?n?n  3*  («1n) 


TABLE  6-1 
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The  A  matrices  are  composed  mostly  of  zeros  except  for  the  diagonal 
submatrices  Uk>  Finally,  observe  that  c<),  the  cost  associated  with  the 
binary  variables,  Is  zero  In  this  case,  a  fact  which  will  further 
simplify  calculations. 


6  11  SOLUTION  NuTHOD  BY  PARTITIONING 

In  this  section  we  discuss  the  partitioning  technique  of  Benders  as 
presented  in  Appendix  A  when  it  Is  applied  to  our  multi  staged  link  addi¬ 
tion  problem  M,  We  shall  use  approximately  the  notation  of  Appendix  A. 
In  the  present  case,  the  set  SQ  is  defined  by  all  yijk  satisfying  con¬ 
straints  (6  12),  (6  13),  and  (1.15).  At  each  step  of  the  algorithm,  and 
once  the  auxiliary  (0-1)  problem  3'  has  been  solved  yielding  the  optimal 
values  yjjk,  these  will  be  used  to  solve  each  one  of  the  subprograms: 


Pk  ;  Minimize 

N  a 

2  ~  E  E  c  -  . .  Xi  . . 

a*!  (1,J)eA  1jlc  1Jk 

(6-17) 

Subject  to 

®1j  Xijk  =  r?k’  ¥  a 

(6.13) 

x?jk^u1jk  *  eX 

(6.19) 

x?jk  -  uijk  ^1  jk  ’  ^1,j)  e  Y 

(6.20) 

xi  jk  -  °*  ^  E  A*  ¥  a 

(6.21) 

for  all  values  of  k  (all  time  periods).  However,  problem  Pk  is  essentially 
a  multi-copy  network  flow  problem  and  therefore,  at  each  iteration  of  the 
partitioning  algorithm,  n  problems  of  the  form  must  be  solved  for  the 
given  values  y°ik  Each  of  these  problems  presents  in  turn  a  block- 
anguiar  structure  and  thus  j  higher  ^evel  of  decomposition  may  be  applied- 
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For  example,  the  Dantzlg  and  Wolfe  decomposition  method,  as  Investigated 
by  P innel 1  [8]  may  be  applied.  Whan  this  scheme  Is  used,  the  solution  of 
each  subproblem  at  each  Iteration  of  the  decomposition  procedure  reduces  to 
finding  its  shortest  path  r/ee. 

As  for  the  second  part  of  the  partitioning  algorithm,  the  solution 
of  all-integer  (0-1)  programming  pr.blen;  Is  required  with  one  or  two 
additional  constraints  at  each  iteration. 

The  problem  presents  the  following  form: 


B'  :  Min  mize  i  t  y. 

k  =  1  K 


(6.22) 


Subject  to 


i?1  "ikt  fTk  ’  uijk  ’  .1t^eY  #1jkt  uijk  y1jk  51  yk,  (6*23) 

t  e  T, 


yijk  •  yijk-l  S  0  •  £Y  .  k-1 . "-1 


(5.24) 


(i  j)-y  3lJk  (y>1k  "  -  Bk’  ¥  k 


(6.25) 


y  k  =  0  or  1  ,  ¥  (1,J>  e  Y.  ¥  k 


(6.26) 


y.  unrestricted,  ¥  k 


(6,27) 


where  L^1kt»  are  the  components  of  an  extreme  point  t  c  T^  of  the 

polytope  associated  w>th  the  dual  of 

For  the  one  copy  case  ( n  -  1 )  and  for  integer  demands  and  capacities, 
network  flow  theory  shows  that,  the  dual  variables  r  are  also  integers. 
Hence,  from  (6  ?'),  the  yk  are  integers.  Problem  B’  ■'  therefore  an  all- 
Int-iger  program  from  which  u  feasible  solution  is  already  available  (namely, 
y,  .  -  0,  (i,j)  c  l.  11k)  and  the  Yong-Genzalez  algorithm  may  be  applied. 
For  the  multi-copy  case,  however,  the  values  are  not  necessarily  integer 
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and  the  may  not  be  integer.  B1,  in  this  case.  Is  a  mixed-integer 
linear  program, 

We  conclude  with  the  following  remark  on  proc'ems  with  a  more  flexible 
budget  structure 

REMARK.  The  basic  model  considered  in  formulation  M  may  be  extended  for 
the  case  of  budget  ceiling  deferrals.  The  mechanism  that  permits  funds 
which  remain  unused  in  a  certain  period,  to  be  transferred  to  a  later 
period  is  given  by  a  proper  manipulation  of  the  slack  variables  of  con¬ 
straints  (6.13) 

Let  s  ,  k  ••  1,  n  be  the  amount  of  unused  funds  of  period  k.  The 
constraints  (6,13]  take  the  new  form 


o,j)E»a,Jk 


t'uk  • y 


Jk  J'ijk-I'  k-1 


)-s. 


+  sk  =  Bk- 


k=1 . n  (6.28) 


with  sQ  =0,  and  with  sR  representing  the  idle  funds,  if  any,  at  the  end 
of  the  assignment.  We  observe  that  no  present  worth  factor  is  attached 
to  the  variables  sk,  so  they  represent  for-all  cases,  simply  idle  cash. 

The  solution  to  M  with  (6.28)  instead  of  (6.13)  is  not  substantially 
altered.  Its  influence  will  be  reflected  exclusively  in  the  solution  of 
the  integer  program,  B' ,  by  augmenting  the  problem  with  the  n  slack 
variables  sn<  If  a^  and  are  assumed  to  be  integer,  then  sR,  will 
also  be  integer 
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THE  DECOMPOSITION  PRINCIPLE  FOR  MIXED-VARIABLE  PROGRAMS 

A. 1  INTRODUCTION 

In  tm$  Appendu  we  shall  present  in  detail  the  partitioning 
technique  developed  by  'lender  [)]  is  app'iei  to  bo'h  *ontinuous  and  mixed- 
integer  linear  programming  problems  presenting  the  so-called  block-anguiaA 
structure.  In  the  case  of  continuous  linear  programs  the  method,  as  noted 
by  Benders  and  also  by  Balinskl  [2],  constitutes  a  dual  form  of  the 
Dantzig  and  Wolfe  decomposition  principle  [3].  We  shall  explicitly  show 
this  property 

In  the  linear  programming  case,  the  partitioning  technique  requires 
the  solution  of  a  linear  program  differing  from  the  one  of  the  previous 
iteration  by  one  or  two  constraints.  Thus  the  duaE-acmpEex  method  is 
indicated  to  reoptimize  the  problem  subject  to  the  additional  constraint(s) . 

For  the  case  of  the  (0-1)  mixed-integer  linear  program,  the  parti¬ 
tioning  technique  requires  the  solution  of  an  all-integer  (0-1) 
programming  problem  augmented  at  each  iteration  by  one  or  two  additional 
constraints.  The  Voung-Gonz5lez  algorithm  [4],  [5]  is  the  method  we  have 
selected  for  the  solution  of  the  all-integer  program,  and  a  procedure  to 
reoptimize  the  solution  of  the  previous  iteration  is  indicated 

The  partitioning  technique  developed  here  is  directly  applied  in 
chapters  V  and  VI  to  the  solution  of  multi-stage  network  synthesis  problems. 

A. 2  PROBLEM  FORMULATION  CER.VAUON  OF  AN  EQUIVALENT  PROGRAM 

Ue  shall  consider  the  class  of  mathematical  programing  problems  with 
the  following  analytical  formulation: 


A- 1 


A- 2 
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Determine 

X°  and  X? 
-0  —1 

so  as  to 

Minimize 

Z  1  X, 

-0  -0 

m 

r  1  it  i, 
i-l  1  1 

(A.l) 

Subject  to 

\>  Jo 

‘*0 

(A. 2) 

Ai  So  '  Bi 

I,  *  b,  ,  1-1,.  ,  m 

(A. 3) 

l* 

IV 

o 

,  i-l ,  ,  m 

(A- 4) 

X  e  5 
~~0  0 

(A. 5) 

where  Aq  is  an  lmo  *  nQ)  matrix,  A(  is  an  (m1  x  nQ)- matrix,  Bi  an 
(m.  x  n^)  matrix,  and  are  vectors  with  n^  components  and  are 
vectors  with  components  The  nQ  component  vector  x^  is  defined  over 
the  region  5Q  We  snail  define  SQ  as  the  Intersection  of  (A. 2)  and  (A. 5) 
for  the  following  specia’  cases  of  5Q: 

1)  5„  the  non-n°qative  orthant  Therefore 
o 


S  --  (x  /  An  x  -  h  ,•  x  >  0) 
0  —o  0  -o  —o  —0 


(A. 6) 


ii)  5q  the  discrete  set  defined  by  the  vertices  of  the  unit  hyper- 
cube  Therefore 


(* 


0  “0 


V 


oj 


0  or  1,  j  =  l, 


no} 


(A. 7) 


For  case  i),  problem  A  becomes  a  linear  program  In  standard  form  that  may 
be  solved  by  applying  the  decomposition  principle  of  Dantzig  and  Wolfe  to 
its  dual  program  for  case  u),  the  resulting  program  is  a  (0-1 )  mixed 
Integer  programming  problem.  If  each  A  -  0,  l  *  0,1,-.  ,  m  the  problem 
reduces  to  a  set  or  ■ ‘\1< pundit  problems  of  the  form 

Min  z.  -  c  x,  ,  B  x.  -  d,  ,  x  '  0  In  any  case,  the  constraints  of 
problem  A  present  the  rol lowing  associated  structure  of  block-angular 
form: 
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A-3 


(A. 8) 


we  shall  now  deve.^p  a  program  equivalent  to  problem  A  Denoting  by 
x  =  [x, ,  x^,=  ^3  a  solution  vector  to  A,  the  problem  may  be  expressed 
in  the  equivalent  form 


Minim-: 


[.  m 
c  x^  mm  (  I  c  x 

.  r0-0  i  ’■> 


1  Bt  \  V  i, 


i,  >  0)J 


(A.9) 


The  minimization  problem  within  cuAZy  brackets,  for  a  given  value  of 
2^  e  S0,  becomes  a  standard  linear  program  in  x,  which  we  denote  as  P. 
Note  that  solving  P  is  equivalent  to  solving  the  following  set  of  m 
mutually  independent  ■ inear  programs  and  summing  up  their  objective 
function  values: 


P(  :  Minimize  Z1  -  c(  x( 


Subject  to  B(  b,  -  A1  ^  ;  1  1 ,  ,  m 


x  -  0 

—  1 


Problem  P  has  an  associated  dual  program  D  tiiat  may  be  decomposed 
Into  the  following  set  of  sub^iog tiwa  corresponding  to  the  dual  programs 


of  Pi : 
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D1  ;  Maximize 


‘-Si  -  A1  So1'  ii 
Subject  t3  B(  u,  <  i’ 


u(  unrestricted 


where  the  symbo'-  \‘)  indicates  transpose  By  the  duality  theorem  of 
linear  programming,  ’f  feasible  solutions  e*ist  for  both  P1  and  D1 ,  then 
their  optimal  solutions  z°  and  z°  satisfy  -  zj1 

Hence,  expression  ;A  9)  may  De  expressed  in  terms  of  the  dual 
problems  D?  as  follows. 


Minimize 

x  t  s 

-0  0 


|"w  *  ria*  u*i  -  Ai  V  -i  '  Bi  4  ^}] 

L.  i A  <  u  J 


(A. 10) 


Consider  the  convex  polytope  (a  finite  number  of  closed  halfspaces) 

Observe  that  S1  Is  independent  of  the  values  of 


si  *  {y.,  /  BJ  u,  <  & *t > 


x^.  We  shall  assume  temporarily  that  S1  is  bounded  (i  e  ,  it  Is  a  convex 
polyhedron)  Then  from  (A. 10)  note  that  for  any  vatue  of  e  SQ,  the 
maximum  of  each  subprogram  will  occur  at  an  extreme  point  of  S . .  Denote 
by  u^,  k  e  the  extreme  points  of  the  polyhedron  BJ  <  c|.  We  shall 
assume  that  there  are  N(  such  extreme  points  Hence  It  suffices  to  cal¬ 
culate  the  values  of  (b  -Ax)'  u  for  each  extreme  point  and  select 

•  *  ’’"I 

the  maximum  value,  yielding  the  solution  to  for  a  given  value  of  x 
from  the  above  discussion,  (A  »0)  is  equivalent  to 


Minimize 

x  c  S 
-o  o 


[ 


■Hj  -0 


m 

).  ma»  11^ 
>  I  k- 


A,  V 


'  “,*>] 


(A  11) 


Let  y  *  max  f(b  -  A,  *  )'  m  )  ;  then  for  each  extreme  point 

1  kfc-K  1  J 

i 

^1k  the  condition  ho'ds: 


’  A.  V'  ^k  1  *1 


(A. 12) 
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If  we  proceed  analogously  for  all  extreme  points  of  each  S^,  then 
(A.11)  may  be  explicitly  written  In  the  form: 

B  :  Minimize  z  *  c.  <  +  I  y,  (A.13) 

1*1  1 

Suoject  to  (b(  -  At  k^)'  uj)(  <  yt  ,  Mt  ,m,  keKj  (A.14) 


y1  unrestricted  ,  1=1,...,  m  (A. 16) 


The  relationships  !,A  )3)  to  (A  16)  define  a  new  program  in  terms  of 
the  variables  x^  and  y^,  which  we  shall  now  show  to  be  zqiUvalejvt  to 
problem  A, 

If  (aj.  x°)  is  an  optimal  solution  to  A,  then  x?  Is  an  optimal 

solution  to  P.  for  x  -  Also,  D,  will  have  an  optimal  solution  for  a 
i  -o  n~o  i  r 

certain  extreme  point  uT^-  Hence,  In  problem  B,  expressions  (A.14)  will 
be  satisfied,  and  those  corresponding  to  ujk  will  be  satisfied  as  strict 
equalities,  thus  y°  -  (b^  -  A.  x£)'  =  c^  x?.  This  implies  that  the 

optimal  solution  to  B,  (x^,  y9),  gives  the  same  value  fcr  the  objective 
function  as  that  one  obtained  by  A. 

Conversely,  assume  that  (x^,  y°)  is  an  optimal  solution  to  B.  Then 
for  each  i,  at  least  one  value  of  u^  will  satisfy  (A.14)  as  a  strict 
equality,  say  corresponding  to  the  extreme  point  optimal  solution  of 
D^.  Then  by  solving  the  prob'ems  for  x^  *  x^,  we  will  obtain  x?  with 

rn  n  fn  a 

l  £i  Since  ..ne  optimal  solution  to  A  for  a  value 

x?  corresponds  to  the  solution  of  P,  ,  it  follows  that  (x°,  x?)  is 
the  optimal  solution  to  A  with  the  same  value  of  the  objective  function 
as  the  one  obtained  for  B 


For  a  formal  proof  considering  the  unbounded  as  weil  as  the 
infeasible  case  the  reader  is  referred  to  the  work  of  Benders  [1]. 

We  shall  focus  cuf  attention  upon  the  solution  of  problem  B  'or  the 
two  special  sets  defined  at  the  beginning  of  the  section 


A-S 
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A. 3  THE  CASE  WHERE  $Q  [>  THE  N0h-N£6AT.tfE  OR1HANT 

We  consider  pob'em  8  w'th  SQ  given  by  the  expression  (A  6) 
Problem  B,  which  we  shaf  caP  the  mastiii  fnigiam,  becomes 

B  :  M’n’irze  1  -  v,  *  £  y.  ,  -i*),.  ,m  (A  17) 

—0  “0  t  1 

SuD.ect  to  •' a;  ulK)‘  ♦  y,  >  bj  Gjk,  ,N,  IA.18) 


xQ  >  0  ,  yf  un restricted 

Thus,  in  passing  f'om  •’he  linear  program  A  in  standard  form  to  the 
equivalent  tx.'ui-.  oijgt am  B,  by  partitioning  the  set  of  variables  Into 
and  the  remam’ng  «  we  have  reduced  the  number  of  --arables  from  ?  n, 

i  “0  1 

to  (n  ♦  m).  At  the  >ame  time,  we  ha»e  increased  the  numoer  of  con- 

0  m  m 

stramts  from  L  m,  to  (m  »  £  N  1 

.-0  0  ’ 

Direct  solution  of  p  obiem  B  ha-dly  p'oduces  a  positive  net  result, 
since  such  an  approath  implies  tne  cacu'at’-on  in  advance  of  all  con¬ 
straints  of  type  A  ’8),  •  e  th*  cjlcu'jt’o'i  o*  al’  the  extreme  points 
of  the  convex  py/ned-?  S (  Moreover,  it  an  optimum  solution  to  B  is  ob¬ 
tained,  sa>  x^.  y^),  tnen  tne  ,j  ution  to  each  of  the  m  hnea-  programs 
P1  f°r  ’s  equ  red  m  o'Jer  to  rind  the  :or'esponding  optimal 

vectors  x_°  ,m) 

Howeve',  -«  rne  optima’  option  to  B  only  a  sunset  of  1A  18)  will 
be  active  ine  Bende  5  a'^o  nnm  ^o'  solution  of  problem  B  (cf  Section 
A. 5)  makes  use  ot  this  m  attempting  to  generate  those  constraints 
(A. 18)  that  dete-m'ne  optima-  ty  ro'  B  The  procedue  so’ves  B  with  a 
small  sub>et  of  constraints  A  >8);  • r  opt  ma  ’ty  is  not  obtained,  the 
aubp-ugiama  D(  a-e  so’ved  to  generate  addifona1  constraints  to  the 
witeo  jcivgijrf!  J,  whi;n  m  tu'n  will  have  to  be  reoptimized  Tnis  al¬ 
ternative  process  's  repeated  unt 1 ’  an  opt-ma'  solution  (if  one  exists) 
is  obtained  m  a  t'"ite  numbe'  of  steps,  gua-anteed  by  tne  act  that  the 
number  ot  const-a  it>  iA  1 8 )  rm:te 
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The  solution  pocedu'e  just  outlined  c’ose’y  parallels  the 
dicompot>\X,*. a i  af.gou.thm  of  Dantzig  and  Wolfe  we  shall  now  interpret  It 
in  detail 

By  obtaining  tne  dual  of  problem  B  we  have 

N. 


C  :  Max'iuze  z  -  l  £  f  X 


i-1  k* 


IK  IK 


.  N< 

Subject  to  L  l  d.„  »  k  w  *  ^ 

^ _  1  ^  — t <  T k  o-o  -o 


(A. 19) 


(A- 20) 


£  Xik  '  1 
k*  ’  1 K 


(A. 21) 


X1K^° 


w^  unrestricted 
-o 


where  f,k  ■  b;  ulk  and  ilk  •  (A;  u>kf 

We  observe  that  problem  c  corresponds  to  the  so-called  ma&teA  or 
zxXAmal  pioblew  J  the  daj.t  of  A,  in  the  context  of  the  Dantzig  and  Wolfe 
decomposition  principle.  This  justifies  the  name  that  we  have  assigned  to 
B  in  describing  Benaers’  decomposition  principle-  In  C,  the  variables 
A^,  ( i *  1  *  ,m;  k-1,  are  weights  romiing  a  convex  combination  of 

the  extreme  points  uik  of  the  polyhedron  5, 

In  the  optimal  solut'on  to  prob  em  C,  only  a  small  subset  of  the 
variables  A.^  will  be  basic  The  Dantzig  and  Wolfe  method  for  solution  of 
the  dual  program  of  A  makes  use  of  this  tact;  it  tries  to  find  the  subset 
of  Alk  that  determines  optima'ity  tor  C  without  examining  all  basic 
feasible  solutions  This  method  f'-st  obtains  a  basiw  feasib  e  solution 
for  C;  if  the  solution  is  no:  optima’,  the  subprograms  D?  are  solved  to 
generate  a  new  co'umn  u  e  ,  p\op.:ii  n'ctut)  that  should  go  into  the 
basts  of  the  master  program  C  This  process  is  repeated  until  an  optimal 
solution  (U  one  exists)  is  obtained  in  a  finite  number  of  steps, 
guaranteed  by  the  fact  that  the  nunte'  of  extreme  ootnts  of  the  subprograms 
D,  is  finite 
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Finally  we  observe  that  for  optimal  solutions  to  S  and  C, 

(assuming  non-degeneracy),  to  each  basic  variable  of  C  thet*  corres¬ 
ponds  an  active  constraint  of  (A, 18) .  ’his  rol’ows  from  the  complementary 
slackness  conditions: 


x?«  > 0  *  w  ik>  4*4*  -bi  iik 


A. 4  EXTENSION  TO  THE  CASE  WHERE  THE  St  ARE  NOT  ALL  BOUNDED 

If  S1  is  ar  unbounded  polytope,  front  convex  set.  theory  we  know  that 
It  possesses  a  nnite  number  of  extreme  points  u^,  k  e  and  a  finite 
number  of  extreme  * ay*  — 1£’  1  e  l,,  emanating  from  certain  extreme  points. 

Hence  It  may  occur  that  for  a  certain  value  of  e  SQ  In  (A.ll),  the 
solution  to  one  of  the  dual  programs  D^  tends  to  infinity  (l.e. ,  problem 
D.j  is  unbounded)  alcr.g  the  half  line 

t  H-t  3  y.ix  +  k  e  ^ L1 ,  <S  >  0). 

The  corresponding  value  of  the  objective  function  may  be  expressed  as 

l\  ‘  ^  *  Ai  5o>'  Hik  +  •  A1  x-o)!  kl 


If  zi  ■*  «*,  »nd  since  d  >  0  ^om  the  previous  expression,  it  follows  that 


(h,  -  A1  j^)1  Gu  >  0 


(A. 22) 


If  is  unbounded,  problem  P  and  thus  problem  A  are  Infeasible  for 
values  of  e  SQ  for  which  (A. 22)  holds.  Hence,  to  prevent  ^  from  tak¬ 
ing  on  such  values,  it  suffices  to  restrict  (A.ll)  or  its  equivalent 
problem  8  with  the  following  constraints  associated  with  all  extreme  rays 
of 


^  "  A1  *o)'  -U  -  0  1  1  e  L1  » 


(A. 23) 
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The  extreme  rays  of  5.  may  be  obtained  by  finding  the  extreme  rays 
of  ''ts  associated  pclykcdial  corner  com  (u^  /  Bj  ji1  <  0),  In  this  manner 
the  set  is  determined  and  the  set  of  constraints  (A. 23}  may  be 
constructed. 

If  constraints  (Ac 23)  thus  constructed  are  added  to  problem  B,  then 
any  feasible  solution  (x^,  yj  to  B,  with  then  used  In  sc’vlng  the 
problems  P  ,  will  result  in  a  feasible  solution  Qc.,  x, )  to  the  original 
problem  A. 

Finally  consider  the  linear  programming  case  of  section  A. 3  with 
constraints  (A.23)  included,  Note  that  the  resulting  dual  program  C  is 
analogous  to  tne  one  obtained  by  applying  the  Dantzig  and  Wolfe  method  to 
the  dual  of  a  problem  A  containing  unbounded  subprograms. 


A. 5  THE  PARTITIONING  ALGORITHM  OF  BENDERS 

The  Benders1  partitioning  algorithm,  instead  of  solving  directly 
problem  B,  solves  iteratively  a  less  restricted  problem  B‘  with  the  same 
objective  as  B.  At  each  new  iteration,  additional  constraints  are  added 
to  B1  and  the  problem  is  reoptimized.  Since  eventually  B1  would  be  iden¬ 
tical  to  B,  the  optimal  solution  to  the  latter  must  finally  be  found. 
However,  the  method  tries  to  reach  optimality  for  B  by  solving  problems 
B‘  with  only  a  small  subset  of  the  total  number  of  constraints. 

Let  I  be  the  set  of  indices  1-1,.  ,  m,  and  I*  a  subset  of  I.  Also 

let  be  a  subset  of  the  set  of  extreme  points  to  subprogram  1,  and 
L{  Cl(.  Then  problem  B'  may  be  formulated  as  follows: 


Determine 

x° 
— o  ’ 

y°  and  2 

0  so  as  to 

Minimize 

z  * 

cA  i  ♦ 

m 

Z  y. 

-0  -o 

t*l  1 

Subject  to 

% 

-  Al  4' 

•  u1k  <  ;  1  t  r.  k  e  k;  (a.24) 

(b,  -  A1  xo)*  ut/  <  0  ;  i  c  I1,  I  t  L;  (A. 25) 

t  So,  y (  unrestricted;  i»l . ,  m 
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Assuming  that  an  optimal  solution  with  value  2°  exists  for  B,  and 
that  2°  Is  an  optimal  solution  to  B',  It  follows  that  2°  <  z°,  since  B' 

Is  restricted  than  problem  B.  That  is,  the  solution  to  B'  Is  a  lower 

bound  on  the  optimal  solution  to  B 

The  solution  process  consists  of  solving  B‘  and  obtaining  (x?,  y?) 
and  2°  »  c.  t  T.  y<,  (if  Bs  is  infeasible  then  it  follows  that  B  is 

—0  — 0  •'i 


infeasible),  Tne  t"al  so'ution  x®  is  then  »ep*  aced  tn  B  and  the  problem 
is  solved  for  the  ^a’ues  of  y^  Solving  B  to'  a  given  x^  *  is  equiva¬ 
lent  to  solving  tne  subprograms  D(  for  that  value  of  and  c  te ini ng  a 

set  of  extreme  po’nti  j  and  a  set  of  values  y°  Thus  problem  B  for 

x.  =  has  the  solution  y°),  and  z  *  c^  x:  ■*  L  y°.  Now  z  and  2° 
-0-0  1*1  1 


are  compared  If  2°  <  z,  then  the  current  constraints  of  B'  do  not 
determine  optimality;  new  constraints  generated  from  the  extreme  points 
obtained  from  the  solution  to  Di  are  therefore  added  to  B'  to  complete 
one  iteration  of  the  algorithm  On  the  other  hand,  if  2°  *  z  the  solution 
2°  tu  B'  is  optimal  for  B,  and  tnus  it  satisfies  the  original  problem  A. 

Next  we  shall  restate  the  algorithm,  considering  all  of  the  different 
situations.  For  a  rigorous  proof  of  the  termination  rules,  the  reader  Is 
referred  to  the  work  of  Benders,  [1]. 


INITIAL  STEP.  Obtain  a  subset  of  extreme  points  and/or  extreme  rays 
L'  to  generate  problem  B'. 

STEP  a.  Solve  problem  B‘ 

a.l)  If  B'  is  infeasible,  8  is  infeasible;  stop,  problem  A  has  no 
feasible  solution 

a. 2)  If  B'  is  unbounded  below,  take  as  the  value  of  x^  for  step  b 
any  feasible  i  of  B'  corresponding  to  a  small  value  of  2. 

a. 3)  Otheiv  se  B'  has  an  optimal  solution  2°  and  (^,  y°),  so  go  to 
step  b 

STEP  b.  Solve  problem  B  for  ^  from  step  a.  That  is,  solve  all 
subprograms  Dj  for  that  value  of  x p. 
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b,1)  If  one  of  the  subprograms  D1  is  Infeasible,  stop.  Problem  A 
is  euher  infeasible  or  unbounded  below,  (This  case  may  occur 
only  during  rhe  fust  iteration  ) 


b.2)  For  each  subprog -am  Qt  that  is  unbounded  above  along  the  hal.f 
line  (u^  /  u1  -  u1(<  +  X  u^},  a<*d  for  a  constraint  of  the 
form  (A  25)  to  problem  8'  ff  also  (b^  -  A^  x^)'  u^  >  y°, 
then  01k  defines  a  constraint  or  the  form  (A, 24)  to  be  added 
to  B*  Go  to  step  a 

b.3)  Otherwise  all  D  have  an  optimal  solution  u^:  Calculate 
v°  -  i  b  -  A  ?  i '  u°  or1  1  o  ^r»<) 

i  l  t  —  u 


STEP  c 


Obtain  i 


-o  -o 


m 

t 

i*l 


o 


c.l)  If  2°  =  z,  stop;  the  solution  z  and  (>[£,  y°)  is  optimal  for  B. 
By  obtaining  the  solutions  to  the  problems  we  obtain  x°, 
and  thus  z  and  (x^,  x°)  constitute  an  optimal  solution  for  A. 

c,2)  If  2°  <  z  then  each  o*  the  u?k  defines  a  new  constraint  of  the 
form  (A  24)  to  be  added  to  B'-  Go  to  step  a. 


We  note  the  following  properties  of  the  algorithm: 

1)  Each  time  that  stop  a  Is  executed  (l.e. problem  B'  solved), 

2°  constitutes  a  lower  tourd  on  the  optimal  solution  z°  which  is  also  a 
better  lower  bound  than  that  of  the  previous  Iteration. 

11)  Whenever  step  b  is  executed  (l.e.,  problem  B  is  solved  for 
x°  *  X^),  either  we  obtain  the  optona.1  solution  to  B,  (detected  by  c.l) 
or  the  solution  to  B  constitutes  an  upper  bound  on  z°,  (l.e  , 
z°  <  c  2°  +  ?  y?).  The  best  upoer  bound,  however,  does  not  necessarily 

‘  “°  0  i.i  ’  — 

correspond  to  tne  value  of  the  objective  function  cf  B  obtained  in  the 
current  iteration,  but  is  obtained  as  the  minimum  value  of  the  objective 
function  of  B  over  all  iterations  performed  so  far 


A. 6  THE  CASE  WHERE  $„  IS  THE  SET  OF  VERTICES  OF  THE  UNIT  HYPERCUBE 

o 

In  this  section  we  consider  problem  B  In  a  slightly  different  form, 
in  order  to  conveniently  study  the  case  where  So  is  given  by  expression  (A.  7) 
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In  expression  (A, 11),  set  yQ 


c.  X-  ♦ 

-o  -o 


< !,  It,  -  N  s.)'  “V‘ 


then  for  any  combination  of  m  extreme  points  taken  one  from  each  convex 
set  S^,  the  following  condition  holds! 


^0 


So 


m 

Z 

1«1 


-  A,  *o>'  i,k  *  »0  1  k  6  Kl 


Expression  (All)  may  therefore  be  conveniently  expressed  in  the 
form  of  the  equivalent  problem  to  A: 


6  ;  Minimize 

2 

(A. 26) 

Subject  to 

m 

Z  (b,  - 
1-1  -1 

Alio)’ 

A 

% 

S  V 

,  1 

•V 

...m;  k  e  Ki 

(A. 27) 

%  -  A1 

<0,1 

=1  >• 

. .  ,m;  l  e 

(A. 28) 

Ao*o- 

So 

(A. 29) 

xo  J  ■  0 

or  1 

(A. 30) 

y„  unrestricted 

(A. 31 ) 

The  (0-1)  mixed-integer  linear  programing  problem  A  Is  thus 
equivalent  to  B.  Except  for  the  unrestricted  variable  yQ,  problem  B  Is  a 
(0-1)  all-integer  programing  problem  with  n0  Integer  variables  and  a 
potentially  large  number  cf  constraints,  f-or  certain  network  flow  problems 
Involving  decision  variables  and  satisfying  som?  additional  Integrality 
conditions  on  the  Input  data,  the  variable  yQ  may  also  be  restricted  to 
be  Integer  (cf.  Chapter  V)?  It  is  for  this  class  of  problems  that  we 
shall  discuss  the  solution  procedure  of  step  a  of  the  Benders  algorithm. 

The  problen  8',  (B  with  a  subset  of  (A. 27)  and  (A. 28),  will  then 
contain  nQ  ♦  1  Integer  variables  and  may  be  solved  by  means  cf  a  branch 
and  bound  algorithm  (cf.  Chapter  II).  However,  we  consider  that  the  primal 
all-integer  algorithm,  developed  Independently  by  Young  [4]  and  Gonzalez 
[5]  and  denoted  here  as  the  Voang -Gonziliz  algo'UXir',  Is  more  suited  to 
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the  conditions  of  the  problem.  Indeed,  the  following  properties  Indicated 
by  Gonziisz  [5]  fully  apply  to  the  solution  of  B* : 

0  Gonzalez  presents  a  special  procedure  for  treating  unsigned 
variables,  which  applies  in  ouf  case  to  the  integer  and  unrestricted 
variable  y  in  S' 

n)  He  also  presents  a  special  way  to  handle  integer  variables 
restricted  to  take  on  the  values  0  or  I 

111)  Given  the  nature  or  the  objective  function,  (i  e, ,  yQ  is  the 
only  variable  and  has  a  positive  coefficient),  the  initial  tableau 
already  satisfy*  opt  ma  ty  t  row  e'e-nents  s  0)  although  it  may  not 
be  primal  reasiole  Therefore  it  suffices  to  apply  the  Gonzalez  pro¬ 
cedure  to  obtain  a  starting  feasible  solution. 

Finally,  we  have  observed  that  each  time  a  new  constraint  Is  added 
to  a  problem  after  step  b  of  the  Benders  algorithm,  problem  B‘  may  be 
reoptimized  simply  by  updating  the  constraint  in  terms  of  the  current 
tableau.  Since  the  slack  of  the  constraint  will  be  negative,  and  It  Is 
restricted  to  be  positive,  we  apply  the  Gonzalez  method  to  remove  the 
infeaslbility  of  the  slack  variable  rhis  operation  may  alter  the 
optimality  of  the  first  row  If  this  i*  Lhe  case,  the  tableau  is  then 
reoptimized  From  the  properties  we  have  Indicated,  we  consider  that  the 
application  of  the  algorithm  In  Chapter  V  for  the  solution  of  the  multi- 
stage  link  addition  problem  is  justified. 


A. 7  SOLUTION  OF  AN  EXAMPLE  PROBLEM 


Consider  the  following  problem: 


A  :  Min  I  -  7*^  ♦  -  5*^  *  4*^  ♦  3*^ 


4 x |  *  2*2  * 

4*,  ♦  3*2  *  Xj 


5* . 


1  X, 


2», 


*  X 


12*6 

>  I 

i8,  f  »,  i  0 
3*6  -  5 
4*e  i  6 
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Although  the  problem  is  not  in  standa'd  form,  we  snail  not.  add  slack 
variables,  but  w'*1  use  the  graphical  representation  ot  the  sets  S... 
Conslde'  two  subprograms' S^  and  Sg,  indicated  »n  Figs,  A-l  and  A-2 
respective'y  rnen  the  various  elements  of  problem  A  are 


*2  *3  *4  *5  *6 


S2  1  U u3 ru4 j  /  u3  4t  u4  <  3V  -3u3  -  4u4  5  -12;  u3>u4  >  0} 

INITIAL  STEP  We  shall  assume  that  an  extreme  point  for  each  St  is  known, 
say  un  s  (0,0)  ana  u^,  -1  (0,  3). 

ITERATION  1 

STEP  a.  We  solve  B‘.  To  reduce  the  number  of  variables  in  A1  to  2, 

(i  e-  ,  Xj  and  y) ,  thus  penmtt  ,'g  a  graphical  solution  of  this  step, 

2 

we  shall  set  y  :  c  x  ♦  I  man  (b  •  A,  x_)'  u  . . 

~°  ~°  i- l  keKi  ‘  ~° 

B‘  :  Mm  2  =  y 

y  -  >  »8 

X,  1  0  y  unrestricted 

By  inspection  (see  F-g  A-3i  the  optimal  solution  is  H°  ■  18,  and  X°  1  0. 

STE3  b.  We  so'jc  the  >ubp'og'am$  0,  and  0^  *or  x^  -  X^  1  0. 
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Dj  :  Max  *  u^  +  8u2  Dg  :  Max  z2  *  5Uj  +  6u^ 

2U|  +  3Uo  <  6  S  4 

5Uj  +  Ug  &  5  U4  <  3 

u1 ,  u,  >  0  -  3u3  -  4u^  <  “12 

u3»  u4  >  0 

D^  :  optimal  so  V  t  *  on  2®  16,  £j2  -  (0,  V. 

D2  :  optimal  solution  z 2  -  38,  -  (4,  3} 

STEP  c.  We  obtain  z  for  the  current  value  of  x^:  z  *  7.  >E°  +z'j  +  z2  «  54; 

therefore  'z  *  18  <  z  =  54,  and  the  optimal  solution  z°  Is  bounded 

as  follows:  18  s  z°  <  54 

ITERATION  2 

STEP  a.  6'  :  Min  2  =  y 

13 

y  +  23  9^  >54  (new  constraint) 

?^  >  0,  j)  unrestricted 

Optimal  solution,  (see  Fig.  A-4):  !°  3  19.5,  and  -  1.5 
STEP  b.  We  solve  and  D2  for  X|  *  X®  *  1.5 

D|  :  Max  Zj  *  -5u^  ♦  2u2  D2  :  Max  z.,  ®  -2.5u.j  ♦  3u^ 


(uj ,  u2)  e  S1 


{ ui  i ,  ^ ^2 
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D1  :  optimal  solution  -  4,  :  (0,  2) 

D2  :  optimal  so-ution  -  9,  -  1.0.  3) 

STEP  c.  2  -  7X°  -  2°  ♦  2°  -  23  5;  therefore  2°  -  19  5  <  z  --  23.5  and  the 
updated  bounds  for  z°  are  19.5  <  20  <  23.5. 

ITERATION  3 

STEP  a.  B'  :  Min  2  *  y 

y  *  7  >16 

y  ♦  23X,  >  54 

"  7x^  >  34  (new  constraint) 

a  ",  y  unrestricted 

Optimal  solution,  (see  Fig.  A-5):  2°  *  20  and  *  2 
STEP  b.  We  solve  D1  and  D2  for  x^  *  X®  *  2 

D-|  :  Max  2 1  *  -7u.  O2  :  Max  22  11  -5u^  ♦  2u^ 

'  u2^  •  1^2 )  ^  ^2 

0^  :  Optimal  solution  2°  *  0,  and  either  ■  (0,0)  or  u12  *  (0,2) 

Dj  :  Optimal  solutton  2))  4  ^21  *  (0*3) 

STEP  C;  2  -  7  1°  ♦  2°  *  2°*  20,  and  2°  »  z ,  optimality 

Thus  the  optimal  solution  is:  2°  •  20,  x°  *  2,  u°  *  0,  u°  *  0  or  2 
*  0 ,  uj  *3 

To  obtain  me  optimal  values  of  the  primal  variables  x2,  x^, 
x^  and  Xg,  it  suffice  to  solve  the  linear  programs  and  P2  for 
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P1  :  Min  iy  *  6*2  *  5*3 


2*2  *•  ^*3  >  -  1 

3*2  ♦  X3  i  0 


P2  :  Min  z2  *  4x4  +  3xg  -  12xg 

x4  "  3xg  i  -5 

Xj  •  4Xg  i  2 


:  Optimal  solution,  -  0,  x2  *  x^  a  0 
P„  :  Optimal  solution,  zc  *  6,  x°  *  0,  x?  a  26  ,  xg  *  5/3 


Normally  this  step  of  solving  P^  and  P2  Is  avoided  sinceE  for  most 
algorithms,  the  solution  to  problems  Dj  and  D2  also  determine  the  optimal 
solution  to  their  dual  programs  Pj  and  P2» 


FIG.  A- 3 


\  (1.5.  19.5- 


■ 


I G .  A-5 
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HISTORICAL  SURVEY  OF  OPTIMIZATION  THEORY 

B.l  EARLY  DEVELOPMENTS 

It  is  now  generally  accepted  that  the  first  investigations  of 
optimization  were  carried  out  by  the  Greek  geometricians  of  the  first 
Alexandrian  School  (circ  300-30  B.C.),  The  well-lcnown  historian  of 
mathematical  thought,  Moritz  Cantor  [1],  attributes  to  Euclid  (Book  VI, 
prop.  27)  the  first  ex;  ’e  of  an  extremal  problem  in  the  history  of 
Mathematics.  TM  proposition  proves  by  synthetic  means  that  if  a  straight 
line  segment  ’s  divided  inro  vwo  parts  the  product  of  both  parts  is  maxi¬ 
mum  when  the  parts  are  equal  No  less  familiar  to  Euclid  were  the 
following  prcMems:  '1  he  perpendicular  is  the  minimum  among  all  straight 
lines  that  may  be  drawn  from  a  point  to  a  line  "  and  "The  diameter  or  a 
circle  is  the  maximum  among  all  inscribed  lines." 

The  othey.  twc  geometers  who  share  with  Euclid  the  fame  accreditee  fo 
the  Greeks  of  that  school  are  Archimedes  and  Apollonius  of  Perga,  who  also 
were  concerned  with  problems  of  maxima  and  minima.  The 'former,  in  tne 
second  book  of  Ms  work  on  the  Sphere  and  Cytinde.fi,  proposes  the  following 
"of  all  spherical  segments  whose  surfaces  are  equal  the  hemisphere  has  the 
greatest  volume".  The  latter,  celebrated  for  his  work  on  the  conic  sec¬ 
tions,  determines  in  his  fifth  book  "the  shortest  line  that  may  be  drawn 
from  o  point  to  a  given  conic  section". 

Pappus  of  Alexandria,  who  belongs  to  the  second  ,r1exandnan  school 
(30  8.C.  -  641  AD  )  is  credited  with  the  solution  of  several  "isopheri- 
metric  problems".  The  first  ten  propositions  of  his  fifth  book,  [3],  ore 
directed  towards  the  proof  of  the  proposition  that  among  all  figures  of 
sane  perimeter,  the  circle  has  the  greatest  area,  he  later  r-?marks  that 
if  most  of  the  properties  of  the  sphere  had  already  been  found,  one 
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remained  to  be  proven:  "of  all  the  solid  figures  having  the  same  area 
the  sphere  has  the  greatest  volume"  His  proof  ’S  not  general  and  the 
problem  became  one  of  the  most  controversial  issues  of  mathematical  history. 
Its  rigorous  solution  was  not.  obtained  until  the  second  half  of  the 
eighteenth  century  Dy  means  of  the  calculus  of  variations. 

In  his  book  VII,  Proposition  61,  Pappus  solves,  by  synthetic  geometry, 
tne  following  problem:  "Minimize  the  function  his  solu- 

tion  is  much  simpler  and  more  elegant  than  its  analytic  counterpart  using 
the  differential  calculus 


In  the  seventeenth  century,  after  a  deep  ’apse  of  mathematical 
progress  characteristic  of  the  Middle  Ages,  and  before  Newton  and  Leibnitz 
developed  the  calculus,  Fermat  published  his  Methodic;  ad  dL&qiu.wndum 
maximum  ef  tnuumum  8all  [2]  suggests  that  his  method  was  developed  after 
a  remark  by  Kepler,  that  the  values  of  a  ^unction  in  the  neighborhood  of 
an  extreme  point  on  either  side  must  be  equal.  He  solves  the  problem 
treated  by  Euclid  of  finding  two  numbers  such  that  its  sum  is  qiven  and 
its  product  is  to  be  minimized.  His  method  is  equivalent  to  taking  a 
neighboring  value  of  x,  namely  x  +  e>  where  e  4s  v?ry  email,  and  setting 
x(a-x)  *  (x+e)  (a-x  -  e).  Simplifying  the  algebra  and  ultimately  setting 
e  *  0,  the  solution  is  obtained  for  x  -  j.  Later  Huygens,  from  the  Hague, 
stated  in  general  terms  the  rule  used  by  Fermat.  About  1573  he  solved 


the  problem:  two  points  Pj,  Pg  net  on  the  straight  l.ne  AB  are  given. 
Find  a  point  P  on  AB  such  that  ♦  vf  is  a  minimum. 


B.2  CLASSICAL  PERIOD 

The  epoch  of  formal  development  of  c’acsical  optimisation  theories 
(indirect  methods  based  on  the  differential  calculus)  begins  with  the  In¬ 
vention  of  the  calculus.  The  theories  obtain  necessary  conditions  to  be 
satisfied  by  an  optimum  point  Sufficiency  was  seldom  satisfied  and  new 
means  to  prove  it  remained  to  be  discovered.  The  main  contributors  were 
Newton,  who  applied  his  m?tkvd  ofl  & to  problems  of  maxima  and 
minima,  and  le’bmtz,  who  published  in  his  A  eta.  E\uiiux>\jm  of  Octob  .*  lt>84 
a  general  method  for  finding  maxima  and  minima 
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During  the  second  pa<t  of  the  eighr/enth  century  a  large  class  of 
optimization  problems,  the  optimization  of  a  uefimte  integral,  was  studied 
by  the  Bernoulli  and  Eu'e'  and  systematized  by  Lagrange  This  new  branch 
of  optimization  theory  was  termed  the  Calculus  05  sxxiatuns ,  a  name 
suggested  by  Euler  Previous  developments  in  mechanics  suggested  to  Euler 
that  all  natural  phenomena  present  extrema,  and  his  later  work  constitutes 
an  important  app'icafon  of  optimization  theory  to  mechanical  systems. 

A  complete  account  or  optimization  problems  in  mechanics  can  be  found  in 
[10]-  On  the  so'ution  of  optimization  problems  subject  to  subsidiary 
conditions,  a  systematic  method  was  given  by  Lagrange  in  his  T hlon.ce.  de* 
fonctions ,  which  determines  a  set  of  necessary  conditions  for  an  extremum 
of  a  function  subject  to  equality  constraints. 

In  the  nineteenth  century  the  work  of  Weierstrass  of  the  University 
of  Berlin  served  to  formalize  the  theory  of  maxima  and  minima.  He  was 
primarily  concerned  with  existence  conditions,  which  had  been  somewhat 
disregarded  probably  due  to  the  fact  that  in  many  physical  applications 
either  a  maximum  or  a  minimum  obviously  exists  His  existence  theorem, 
based  on  the  work  or  Bolzano,  states  that:  tjj  a  (.unction  flU)  a>  continu¬ 
ous  <n  a  :  k  .  b,  flute  msts  and  1 2>  a  *  S|  -  -»  1  -  '2  *  ($0*  whuch 

the  function  attains  its  laigest  value  M  and  its  smallest  value  m. 

Jacob  Steiner,  another  mathematician  or  the  University  of  Berlin, 
representative  or  the  geometric  school,  solved  in  the  ecriy  nineteenth 
century  a  prob'em  posed  earlier  by  Fermat,  whicn  lias  had  important  appli¬ 
cations  in  genera' 'zed  for-  to  location  theory  The  problem  is:  given 
three  points  A  B  C  in  a  p'aie,  find  a  fourth  point  P  such  that  the  sum  of 
the  Euclidian  distances  from  each  or  the  three  points  to  P  is  minimized. 
This  problem  has  been  widely  publicized  by  Cou^ant  and  Robbins  [II]  and 
has  lately  been  treated  by  Kuhn  in  [12]  to  present  an  interesting  duality 
concept  in  non'me-i'  programming 

Another  source  of  so'ution  methods  for  optimization  problems  which 
has  proved  effective  is  the  genera'  theory  or  inequalities  (see  for 
example  [6]  or  [>5jl  It  is  worth  mentioning  that  the  application  is  re¬ 
ciprocal:  name'y,  inequality  theorems  may  be  proved  with  the  auxiliary 
solution  of  a  maxim: m  and  minimum  problem,  while  certain  optimization 
problems  may  be  solved  by  the  use  of  known  inequalities  This  reciprocal 
character  is  formalized  by  Chrystal  in  [5] 
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An  inequality  tnat  has  largely  contributed  to  the  method  mentioned 
above  and  that  ha>  been  a  basic  cornerstone  of  the  latest  development  of 
Optimization  tneo-y,  Geomet-'c  Programming,  [7],  is  the  so-called 
giorr.iVnz  This  inequality  states  that  for  a  finite  set  of  non- 

negative  number,  the  a-itnmetic  mean  is  at  least  as  great  as  the  geometric 
mean.  The  English  mathematic < an  MacLau”-m  is  credited  with  the  first 
general  proof  or  tne  geometer  inequality  He  enunciated  the  theorem  in 
the  following  geometric  form,  [8]:  1$  a  Urn  *8  a  cL/xded  cuto  any 

nunbe't  pxrf-.  pijiazt  id  tiut pa*.t~  xk it  be  a  maxonm  uihen  the 
poAtt>  a u  vn.-nq  rhi/nad-cn  rne  best  known  analytical  proof  of  this 

classical  inequa'‘ty,  however,  is  due  to  Cauchy,  [ 1 5 ] 

In  the  work  of  Harris  Hancock  [12],  [1 3 J  published  in  1917,  we  find 
an  excellent  summary  of  what  may  be  considered  the  state  of  classical 
optimization  theory  up  to  that  time  In  [13],  Hancock  indicates  that 
several  inaccuracies  carried  through  from  the  developments  of  Lagrange  were 
corrected  when  a  major  revision  of  the  theory  of  maxima  and  minima,  sug¬ 
gested  by  Peano  or  TUrm,  was  carried  through  by  the  work  of  Scheeffer, 
Stolz  and  Dontscher  in  [13],  sections  109-1 1 2 ,  he  presents  the  treatment 
of  constrained  optimization  problems  subject  to  inequality  restrictions, 
and  makes  use  of  quadratic  s'ack  variables  to  reduce  the  problem  to 
equality  constraints 


8.3  MODERN  PtRiOO  FIRST  DECADE 

It  is  Jj'iny  tne  modem  period  that  the  theory  of  maxima  and  minima 
has  been  widely  o-oadened  and  given  tne  now  generally  accepted  name  of 
•  -i  re.  v,  Primer ’’y  responsible  to'  such  a  task  are  the 
American  s^'entists  ..-0  ca-ned  on  the  development  of  the  theory  during 
and  after  Wor 'd  war  1 1 

The  modern  period  of  optimization  theory  (or  "renaissance",  as  Nemhauser 
[17]  likes  to  put  it),  started  m  1 94 7  with  G  Dantzig's  Simplex  method 
for  the  solut -on  of  hnea--  programs  In  the  two  decades  since  that  event, 
the  development  j*  optimization  theo  y  has  been  e*t"emely  fruitful  In  both 
pure  analytical  re  gues  and  app":ar’ons  to  the  manager! a!  sciences, 
the  military,  engine*,' mg,  jn1  rhe  phy  s  ’  c  a  *  sciences 
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As  in  various  other  fields,  the  substantial  new  developments  that 
took  place  in  the  middle  fifties  are  due  in  great  part  to  the  advent  of 
the  digital  compute'  as  a  common  tool  in  scientific  research  and  develop¬ 
ment,  This  fact  may  very  we' I  lead  in  the  future  to  the  study  of  the 
history  of  sciences  by  dividing  it  into  two  parts:  before  the  computer 
and  after  the  computer  This  is  no  less  true  in  the  case  of  optimization 
theory 

The  first  decade  of  the  modern  period,  i 947-1957 ,  is  characterized  by 
the  formal  solution  of  the  linear  programming  problem  and  the  rigorous 
analysis  of  its  underlying  mathematical  theory  The  work  performed  during 
the  two  yea's  l9i!-’949  was  p-esented  at  the  n:«  histon:  ConmuAion 

con^eacrtcc.  in  Chicago  m  >949  ,  and  selected  papers  were  published  in 
KcUvaAl)  A ia.lyz<.:  0($  Ruda:ti.on  ar,d  ACtocat-con,  edited  by  T.  C.  Koopmans. 

A  number  of  applications  in  business  and  industry  followed,  associated 
with  the  names  of  Charnes  and  Cooper,  who  published  with  Henderson  in  1953 
what  constitutes  the  first  textbook  on  the  subject  matter  [18],  The  book 
of  Gass  [19],  although  published  in  1958,  may  also  be  considered  a  product 
of  the  early  developments  of  linear  programming 

The  principles  of  the  mathematical  theory,  as  well  as  the  statement 
of  duality,  were  la>d  down  by  von  Neumann.  The  rigorous  studies  on  duality 
and  linear  inequality  theories  were  carried  out  and  published  in  the  work 
edited  by  H,  Kuhn  and  A  Tucker  of  the  Princeton  school  in  1S56,  UneaA. 
InequaEctces  aid  Related  Sj/atema 

The  success  and  achievements  of  this  decade  stem  largely  from  the 
development  of  computer  codes  for  the  solution  of  linear  programs  which 
bridge  the  gap  between  theory  and  practice  and  open  a  wide  avenue  of 
applications 

Almost  n  pa*a"el  with  linear  programming,  R  Bellman  [20], 

S.  Dreyfus  [/5]  and  othe-s  have  developed  another  powerful  optimization 
technique,  dyunu:  i-Wru  >ig ,  of  particular  application  to  problems  of 

optimal  control  and  multistage  decision  processes 

For  a  complete  account  of  the  background  of  and  contributors  to  the 
modem  development  of  mathematical  programming,  the  reader  is  referred  to 
Dantzig's  own  account,  [21] 
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8.4  MODERN  PERIOD  SECOND  DECADE 

The  second  decade  of  iiov'irn  period  has  been  rno-'e  prolific  by  far  In 
the  development  of  new  methodologies  of  optimisation  It  has  seen  the 
verification  of  Dantzig’s  prediction  in  his  opening  address  to  the  Third 
Symposium  on  Mathematical  Programming  held  in  Santa  Monica,  California, 
in  1S59,  [22].  "Today,  we  who  are  gathered  here  are  about  to  witness 
the  start  of  an  explosion  11 

We  shall  mention  briefly  the  highlights  of  such  accomplishments  and 
the  principal  contributors  to  each  field. 

The  special  u-umjcfafatUi/  piopnity  of  certain  classes  of  linear 
programs  observed  by  Dantzig  [22],  has  been  a  keystone  of  the  development 
of  network  flow  theory  The  principal  contributors  have  been  Ford  and 
Fulkerson  [23],  [24]  who  proved  the  maximum-flow-minimum  cut  theorem  for 
homogeneous  commodity  flow  in  networks;  Berge  [25]  with  his  rigorous  work 
on  graph  theory;  and  Kuhn  [26]  with  his  work  cn  combinatorics  and  the 
assignment  problem  Generalizations  of  network  flow  theory  have  been  made 
by  Gomory  and  Hu  [27]  on  mul ti-terminal  flows  and  by  Jewell  [28]  on  multi¬ 
commodity  flow  problems 

In  discrete  and  integer  programming,  this  decade  has  seen  the  systema¬ 
tic  development  of  cutting  plane  methods  by  Gomory  [29],  and  the  so-called 
branch  ar.d  bcaii  techniques  by  Land  and  Doig  [31],  Little  et  al.  [30],  and 
Balas  [71],  ( cr  Chapter  II)  For  detailed  information  on  the  subject, 
the  reader  is  referred  to  the  excellent  work  of  Balinski  [32]  which  con¬ 
stitutes  an  exhaustive  survey  of  integer  programming 

Pressed  perhaps  by  the  growing  number  of  applications  with  the  ever- 
increasing  sizes  of  prob  \'s,  particular  attention  was  given,  starting 
around  1959,  tc  the  exploitation  of  special  structures  presented  by  certain 
classes  of  problems  F.-om  these  studies  evolved  the  Decomposition  Principle 
of  Dantzig  and  Wolfe  [33],  without  a  doubt,  a  major  contribution  to  the 
operational  solution  of  linear  programs  Other  types  of  partitioning 
algorithms  have  been  proposed  by  Benders  [34],  3a!as  [69],  Rosen  [70],  and 
others 

In  the  area  of  stochastic  programming,  initiated  by  the  two-stage 
model  of  Dantzig  [35]  and  the  work  of  Tmtner  [36],  much  remains  to  be 
investigated  fne  last  ten  years  have  witnessed  the  work  of  Charnes  and 
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Cooper  and  their  chance  constrained  model  [37]  as  well  as  the  most 
Important  work  of  Madansky  [38],  [39]..  Of  special  interest  in  the  last  few 
years  Is  the  worn  of  Dantzig  [40],  Van  Slyke  [41],  and  Wets  [42]  on  the 
integration  of  mathematical  programing  and  optimal  control  theory,  and  the 
application  of  the  two-stage  approach  of  stochastic  programming  to  the 
latter.  As  Madansky  [43]  puts  it,  the  tak  introduced  into  the  programing 
problem  has  to  do  w’tn  the  probability  distribution  of  the  random  variables 
of  the  problem  when  these  are  completely  known,  The  unczitacnty  arises 
when  the  probability  distribution  is  known  in  form  but  one  or  more  para¬ 
meters  are  unknown  An  important  aspect  likely  to  be  developed  in  the 
future  is  f'e  !•  "c  J . : t *  on  cf  Bayesian  conrepts  in  the  multistage  models, 
providing  the  capability  of  updating  the  probability  distributions  asso¬ 
ciated  with  the  problem  as  more  information  is  available  in  the  process. 

The  remaining  topic,  and  its  basic  theoretical  paper  by  Kuhn  and 
Tucker  in  1951,  [44],  generalizing  Lagrange's  method  for  the  case  of  in¬ 
equality  constraints,  is  the  topic  of  nonlinear  programming.  Approximate 
solution  methods  were  developed  during  the  first  decade  by  Chames  and 
Lemke  [45]  and  Dantzig  [46]  The  special  case  of  quadratic  programing  has 
been  well-studied  by  Beale  in  1955  [47],  Frank  and  Wolfe  [48],  and  Wolfe 
[49].  Other  solution  techniques  of  classical  nature  known  as  gradient 
methods  dating  back  to  Cauchy,  were  consolidated  in  the  early  book  edited 
by  Arrow,  Hurwicz  and  Uzawa  [50]  and  the  later  work  of  Lemke  [51],  Rosen 
[52],  Zoutendijk  [53],  Davidon  [54],  Doerfler  [55],  and  others 

From  the  field  of  numerical  analysis  several  methods  for  unconstrained 
optimization  have  been  developed  in  the  sixties,  and  in  several  instances 
they  have  been  generalized  for  handling  constraints  Of  the  Indirect 
optimization  type  we  mention  the  work  of  Fletcher  and  Reeves  [56].  The 
direct  search  methods  are  based  generally  on  the  work  of  Hooke  and  Jeevis 
[58],  and  the  random  search  methods  on  the  work  of  Karnopp  [59]  and  Brooks 
[60]. 

As  a  final  remark  on  nonlinear  programing ,  we  mention  again  the 
latest  development  that  seems  to  be  a  very  promising  optimization  tool 
for  engineering  design,  constituting  a  generalization  of  the  use  of  in¬ 
equalities  m  the  solution  of  extremum  problems  The  work  has  been  given 
the  name  of  Gcl wvti-c  hiogtiwung  by  its  developers.  Zener,  Duffin  and 
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Peterson,  1967,  [7],  and  It  deals  with  the  optimization  of  unconstrained 
or  constrained  "posynomlals"  (positive  polynomials).  It  has  already  been 
generalized  for  the  case  of  negative  terms  by  Passy  and  Wilde,  [61]. 

In  all,  the  difficult  field  of  nonlinear  programing  has  not  yet 
yielded  to  a  systematic  treatment;  we  feel  that  a  unifying  theory  remains 
to  be  presented 

Of  the  text  books  of  the  second  decade,  we  mention  the  two  books  of 
Hadley  In  linear  and  nonlinear  programming  [62],  [63].  The  latter,  If 
perhaps  not  a  complete  or  perfectly  organized  work,  Is  the  first  general 
text  In  this  area  The  book  of  Dantzig  [21],  that  of  Vadja  [64],  and 
probably  the  best  text  so  far  in  linear  programming,  the  translation  by 
Jewell  of  Simmonard's  textbook  [65],  also  were  published  In  this  period. 
Finally,  we  mention  the  book  of  Wilde  and  Belghtler  [61]  and  the  book  on 
nonlinear  programing  edited  by  Abadle  [67]. 


B.5  FUTURE  RESEARCH 

The  Sixth  International  Symposium  on  Mathematical  Programming  that 
took  place  at  Princeton  in  August  1967,  marks  the  beginning  of  the  third 
decade  of  research  and  development  on  optimization  theory-  From  the  work 
presented  there.  It  is  possible  to  Infer  which  are  tiie  currents  of  research 
likely  to  be  developed  In  the  near  future.  Although  substantial  research 
seems  to  be  underway  In  most  areas  of  optimization  theory,  we  feel  that 
special  effort  is  being  devoted  to  the  following  areas  of  research. 

The  field  of  discrete  linear  programming  Is  very  likely  to  develop 
rapidly,  as  it  is  row  provided  with  a  duality  theory  analogous  to  its  con¬ 
tinuous  counterpart,  developed  by  Balas  [72].  Also,  important  contribu¬ 
tions  have  been  made  by  Balinskl  [73]  on  a  pair  of  related  problems  known 
as  the  mai ujnm  match  and  the  coveung  p\obiem.  Primal  ^ual 

methods  are  therefore  likely  to  be  developed  which  might  be  of  special  use, 
for  example,  in  network  flow  tneory  for  problems  involving  networks  with 
disjunctive  arcs  (1  e.,  flow  either  zero  or  st  upper  bound).  Such  network 
models  are  particularly  suited  for  solving  tne  class  of  problems  treated 
In  Chapters  III  and  IV.  The  author  Is  currently  engaged  in  this  specific 
oroblem. 
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In  nonlinear  programming ,  more  and  more  applications  In  the  context 
of  engineering  design  seem  likely.  Also,  theoretical  extensions  of  geo¬ 
metric  programming  such  as  the  one  presented  by  Avriel  and  Wilde  [74]  on 
stochastic  geometric  programming,  may  be  expected.  The  same  may  be  said 
about  the  important  top^c  cf  control  theory  Finally,  we  feel  that  the 
efficient  exploitation  of  highly-structured  optimization  models  will 
necessarily  lead  to  new  schemes  for  solution  of  large-scale  problems. 

To  close  this  appendix  we  shall  mention  that  the  development  of 
integrated  optimization  systems,  employing  new  computer  tecnnology  and  the 
wealth  of  optimization  techniques  currently  available,  holds  great  promise 
in  the  solut’on  or  U'ge-scaie  optimum  design  problems.  The  need  for 
powerful  synthesis  algorithms  such  as  thosp  mentioned  in  the  introductory 
chapter  of  this  work  will  contribute  to  this  development. 
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